2024-03-24T11:51:14.881 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-24T11:51:14.882 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617975 --description orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} --name teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi --verbose --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617975/orig.config.yaml 2024-03-24T11:51:14.910 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-24T11:51:14.996 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617975 branch: squid description: orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} email: ceph-qa@ceph.com first_in_suite: false job_id: '7617975' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: debug ms: 1 global: mon election default strategy: 3 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: flavor: default sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: squid sha1: f73f1dfd96849300c4322bd7259a3ab37f94c200 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 8903 sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 sleep_before_teardown: 0 subset: 44/64 suite: orch suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_f73f1dfd96849300c4322bd7259a3ab37f94c200/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: f73f1dfd96849300c4322bd7259a3ab37f94c200 targets: smithi006.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD3e/PZKrLah1XZcqTOFzXV/J62SsgJv/6SOlksvbvGVmMmbKA/fKI1isas8RcBuqxEYk12efNzX97pnO5VlY1w= smithi090.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGcb5kFsxPou4rnrqiB0dnqDeVgbhH2uUF4mYd91oTkngMlYK0tYv+Nws3FIshSjS1mFH1Mi+aoqLfB2qx6c67Q= tasks: - pexec: all: - sudo dnf install runc -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-22_21:08:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-24T11:51:14.996 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_f73f1dfd96849300c4322bd7259a3ab37f94c200/qa; will attempt to use it 2024-03-24T11:51:14.997 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_f73f1dfd96849300c4322bd7259a3ab37f94c200/qa/tasks 2024-03-24T11:51:14.997 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-24T11:51:14.998 INFO:teuthology.task.internal:Checking packages... 2024-03-24T11:51:15.021 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'db0330b1e4e2470d52b750e251e55a522b4f7d69' 2024-03-24T11:51:15.021 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-24T11:51:15.021 INFO:teuthology.packaging:ref: None 2024-03-24T11:51:15.021 INFO:teuthology.packaging:tag: None 2024-03-24T11:51:15.021 INFO:teuthology.packaging:branch: squid 2024-03-24T11:51:15.021 INFO:teuthology.packaging:sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:51:15.022 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-03-24T11:51:15.247 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1764.gf73f1dfd 2024-03-24T11:51:15.249 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-24T11:51:15.255 INFO:teuthology.task.internal:no buildpackages task found 2024-03-24T11:51:15.255 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-24T11:51:15.278 INFO:teuthology.task.internal:Saving configuration 2024-03-24T11:51:15.289 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-24T11:51:15.295 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-24T11:51:15.320 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi006.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617975', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-24 11:45:08.341573', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD3e/PZKrLah1XZcqTOFzXV/J62SsgJv/6SOlksvbvGVmMmbKA/fKI1isas8RcBuqxEYk12efNzX97pnO5VlY1w='} 2024-03-24T11:51:15.338 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617975', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-24 11:45:08.340298', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGcb5kFsxPou4rnrqiB0dnqDeVgbhH2uUF4mYd91oTkngMlYK0tYv+Nws3FIshSjS1mFH1Mi+aoqLfB2qx6c67Q='} 2024-03-24T11:51:15.339 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-24T11:51:15.345 INFO:teuthology.task.internal:roles: ubuntu@smithi006.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-03-24T11:51:15.345 INFO:teuthology.task.internal:roles: ubuntu@smithi090.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-03-24T11:51:15.345 INFO:teuthology.run_tasks:Running task console_log... 2024-03-24T11:51:15.421 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7eff2e2190d0>, signals=[15]) 2024-03-24T11:51:15.421 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-24T11:51:15.428 INFO:teuthology.task.internal:Opening connections... 2024-03-24T11:51:15.428 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi006.front.sepia.ceph.com 2024-03-24T11:51:15.430 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T11:51:15.532 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi090.front.sepia.ceph.com 2024-03-24T11:51:15.533 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T11:51:15.623 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-24T11:51:15.630 DEBUG:teuthology.orchestra.run.smithi006:> uname -m 2024-03-24T11:51:15.648 INFO:teuthology.orchestra.run.smithi006.stdout:x86_64 2024-03-24T11:51:15.648 DEBUG:teuthology.orchestra.run.smithi006:> cat /etc/os-release 2024-03-24T11:51:15.706 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="CentOS Stream" 2024-03-24T11:51:15.706 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="9" 2024-03-24T11:51:15.706 INFO:teuthology.orchestra.run.smithi006.stdout:ID="centos" 2024-03-24T11:51:15.707 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE="rhel fedora" 2024-03-24T11:51:15.707 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="9" 2024-03-24T11:51:15.707 INFO:teuthology.orchestra.run.smithi006.stdout:PLATFORM_ID="platform:el9" 2024-03-24T11:51:15.707 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-24T11:51:15.707 INFO:teuthology.orchestra.run.smithi006.stdout:ANSI_COLOR="0;31" 2024-03-24T11:51:15.707 INFO:teuthology.orchestra.run.smithi006.stdout:LOGO="fedora-logo-icon" 2024-03-24T11:51:15.707 INFO:teuthology.orchestra.run.smithi006.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-24T11:51:15.707 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://centos.org/" 2024-03-24T11:51:15.707 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-24T11:51:15.707 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-24T11:51:15.707 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-24T11:51:15.708 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2024-03-24T11:51:15.739 DEBUG:teuthology.orchestra.run.smithi090:> uname -m 2024-03-24T11:51:15.757 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64 2024-03-24T11:51:15.757 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2024-03-24T11:51:15.814 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Stream" 2024-03-24T11:51:15.815 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="9" 2024-03-24T11:51:15.815 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos" 2024-03-24T11:51:15.815 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora" 2024-03-24T11:51:15.815 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="9" 2024-03-24T11:51:15.815 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el9" 2024-03-24T11:51:15.815 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-24T11:51:15.815 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31" 2024-03-24T11:51:15.815 INFO:teuthology.orchestra.run.smithi090.stdout:LOGO="fedora-logo-icon" 2024-03-24T11:51:15.815 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-24T11:51:15.815 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/" 2024-03-24T11:51:15.815 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-24T11:51:15.815 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-24T11:51:15.815 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-24T11:51:15.816 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2024-03-24T11:51:15.839 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-24T11:51:15.848 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-24T11:51:15.853 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-24T11:51:15.853 DEBUG:teuthology.orchestra.run.smithi006:> test '!' -e /home/ubuntu/cephtest 2024-03-24T11:51:15.856 DEBUG:teuthology.orchestra.run.smithi090:> test '!' -e /home/ubuntu/cephtest 2024-03-24T11:51:15.872 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-24T11:51:15.877 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-24T11:51:15.878 DEBUG:teuthology.orchestra.run.smithi006:> test -z $(ls -A /var/lib/ceph) 2024-03-24T11:51:15.913 DEBUG:teuthology.orchestra.run.smithi090:> test -z $(ls -A /var/lib/ceph) 2024-03-24T11:51:15.939 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-24T11:51:16.000 INFO:teuthology.run_tasks:Running task kernel... 2024-03-24T11:51:16.013 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-24T11:51:16.013 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-24T11:51:16.014 DEBUG:teuthology.orchestra.run.smithi006:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-24T11:51:16.014 DEBUG:teuthology.orchestra.run.smithi090:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-24T11:51:16.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T11:51:16.031 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2024-03-24T11:51:16.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T11:51:16.032 DEBUG:teuthology.orchestra.run.smithi006:> uname -r 2024-03-24T11:51:16.090 INFO:teuthology.orchestra.run.smithi090.stdout:5.14.0-361.el9.x86_64 2024-03-24T11:51:16.090 INFO:teuthology.task.kernel:Running kernel on smithi090: 5.14.0-361.el9.x86_64 2024-03-24T11:51:16.090 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum install -y kernel 2024-03-24T11:51:16.091 INFO:teuthology.orchestra.run.smithi006.stdout:5.14.0-361.el9.x86_64 2024-03-24T11:51:16.092 INFO:teuthology.task.kernel:Running kernel on smithi006: 5.14.0-361.el9.x86_64 2024-03-24T11:51:16.092 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum install -y kernel 2024-03-24T11:51:18.254 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-24T11:51:18.635 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-24T11:51:19.713 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-9 - BaseOS 6.6 MB/s | 2.0 MB 00:00 2024-03-24T11:51:20.020 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS-9 - BaseOS 7.4 MB/s | 2.0 MB 00:00 2024-03-24T11:51:20.670 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-03-24T11:51:21.086 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-03-24T11:51:22.363 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - AppStream 31 MB/s | 19 MB 00:00 2024-03-24T11:51:23.771 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-03-24T11:51:28.538 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.1 MB 00:02 2024-03-24T11:51:28.731 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.1 MB 00:01 2024-03-24T11:51:30.132 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 16 kB 00:00 2024-03-24T11:51:30.148 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-03-24T11:51:32.376 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 21 MB 00:02 2024-03-24T11:51:34.735 INFO:teuthology.orchestra.run.smithi006.stdout:Extra Packages for Enterprise Linux 4.6 MB/s | 21 MB 00:04 2024-03-24T11:51:37.343 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-03-24T11:51:39.450 INFO:teuthology.orchestra.run.smithi090.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-24T11:51:39.450 INFO:teuthology.orchestra.run.smithi090.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-24T11:51:39.563 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T11:51:39.568 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T11:51:39.568 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-03-24T11:51:39.568 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T11:51:39.568 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-03-24T11:51:39.568 INFO:teuthology.orchestra.run.smithi090.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-24T11:51:39.568 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-03-24T11:51:39.568 INFO:teuthology.orchestra.run.smithi090.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-24T11:51:39.569 INFO:teuthology.orchestra.run.smithi090.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-24T11:51:39.569 INFO:teuthology.orchestra.run.smithi090.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-24T11:51:39.569 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T11:51:39.569 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-03-24T11:51:39.569 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T11:51:39.569 INFO:teuthology.orchestra.run.smithi090.stdout:Install 4 Packages 2024-03-24T11:51:39.569 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T11:51:39.569 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 76 M 2024-03-24T11:51:39.570 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 123 M 2024-03-24T11:51:39.570 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-03-24T11:51:39.669 INFO:teuthology.orchestra.run.smithi090.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 764 kB/s | 44 kB 00:00 2024-03-24T11:51:39.907 INFO:teuthology.orchestra.run.smithi006.stdout:lab-extras 47 kB/s | 1.7 kB 00:00 2024-03-24T11:51:40.228 INFO:teuthology.orchestra.run.smithi090.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 24 MB/s | 15 MB 00:00 2024-03-24T11:51:40.878 INFO:teuthology.orchestra.run.smithi090.stdout:(3/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 26 MB/s | 33 MB 00:01 2024-03-24T11:51:41.187 INFO:teuthology.orchestra.run.smithi090.stdout:(4/4): kernel-modules-core-5.14.0-430.el9.x86_6 18 MB/s | 27 MB 00:01 2024-03-24T11:51:41.187 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-03-24T11:51:41.187 INFO:teuthology.orchestra.run.smithi090.stdout:Total 47 MB/s | 76 MB 00:01 2024-03-24T11:51:41.848 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-03-24T11:51:42.072 INFO:teuthology.orchestra.run.smithi006.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-24T11:51:42.072 INFO:teuthology.orchestra.run.smithi006.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-24T11:51:42.162 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T11:51:42.169 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T11:51:42.169 INFO:teuthology.orchestra.run.smithi006.stdout: Package Architecture Version Repository Size 2024-03-24T11:51:42.169 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T11:51:42.169 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-24T11:51:42.169 INFO:teuthology.orchestra.run.smithi006.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-24T11:51:42.169 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-03-24T11:51:42.169 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-24T11:51:42.169 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-24T11:51:42.169 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-24T11:51:42.170 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T11:51:42.170 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-24T11:51:42.170 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T11:51:42.170 INFO:teuthology.orchestra.run.smithi006.stdout:Install 4 Packages 2024-03-24T11:51:42.170 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T11:51:42.170 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 76 M 2024-03-24T11:51:42.170 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 123 M 2024-03-24T11:51:42.170 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-24T11:51:42.253 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-03-24T11:51:42.253 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-03-24T11:51:42.675 INFO:teuthology.orchestra.run.smithi006.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 126 kB/s | 44 kB 00:00 2024-03-24T11:51:43.291 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-03-24T11:51:43.292 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-03-24T11:51:45.853 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-03-24T11:51:46.541 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-24T11:51:46.627 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-24T11:51:49.400 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-24T11:51:49.554 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-24T11:51:54.305 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-24T11:51:54.349 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:51:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:52:04.032 INFO:teuthology.orchestra.run.smithi006.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 703 kB/s | 15 MB 00:21 2024-03-24T11:52:19.872 INFO:teuthology.orchestra.run.smithi006.stdout:(3/4): kernel-modules-core-5.14.0-430.el9.x86_6 756 kB/s | 27 MB 00:37 2024-03-24T11:52:20.431 INFO:teuthology.orchestra.run.smithi006.stdout:(4/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 899 kB/s | 33 MB 00:38 2024-03-24T11:52:20.431 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-24T11:52:20.431 INFO:teuthology.orchestra.run.smithi006.stdout:Total 2.0 MB/s | 76 MB 00:38 2024-03-24T11:52:21.118 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-24T11:52:21.536 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-24T11:52:21.536 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-24T11:52:22.571 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-24T11:52:22.571 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-24T11:52:25.165 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-24T11:52:25.786 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-24T11:52:25.873 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-24T11:52:28.777 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-24T11:52:28.909 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-24T11:52:29.487 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:52:29.488 INFO:teuthology.orchestra.run.smithi090.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-24T11:52:29.488 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T11:52:29.531 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:52:29.921 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:52:29.921 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-24T11:52:29.921 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-24T11:52:29.921 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-24T11:52:30.396 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:52:30.397 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T11:52:30.397 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-03-24T11:52:30.397 INFO:teuthology.orchestra.run.smithi090.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-24T11:52:30.397 INFO:teuthology.orchestra.run.smithi090.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-24T11:52:30.397 INFO:teuthology.orchestra.run.smithi090.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-24T11:52:30.397 INFO:teuthology.orchestra.run.smithi090.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-24T11:52:30.397 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T11:52:30.397 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T11:52:30.788 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q kernel | sort -rV | head -n 1 2024-03-24T11:52:30.925 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-24T11:52:30.925 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-24T11:52:30.925 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-24T11:52:30.925 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-24T11:52:30.925 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-24T11:52:30.926 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-03-24T11:52:30.926 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi090.front.sepia.ceph.com, path=None, version=distro) 2024-03-24T11:52:30.926 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q kernel | sort -rV | head -n 1 2024-03-24T11:52:31.005 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-24T11:52:31.006 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-24T11:52:31.006 DEBUG:teuthology.orchestra.run.smithi090:> sudo rpm -qi grub2-tools 2024-03-24T11:52:31.039 INFO:teuthology.orchestra.run.smithi090.stdout:Name : grub2-tools 2024-03-24T11:52:31.039 INFO:teuthology.orchestra.run.smithi090.stdout:Epoch : 1 2024-03-24T11:52:31.039 INFO:teuthology.orchestra.run.smithi090.stdout:Version : 2.06 2024-03-24T11:52:31.039 INFO:teuthology.orchestra.run.smithi090.stdout:Release : 68.el9 2024-03-24T11:52:31.039 INFO:teuthology.orchestra.run.smithi090.stdout:Architecture: x86_64 2024-03-24T11:52:31.039 INFO:teuthology.orchestra.run.smithi090.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-24T11:52:31.039 INFO:teuthology.orchestra.run.smithi090.stdout:Group : Unspecified 2024-03-24T11:52:31.039 INFO:teuthology.orchestra.run.smithi090.stdout:Size : 8264254 2024-03-24T11:52:31.039 INFO:teuthology.orchestra.run.smithi090.stdout:License : GPLv3+ 2024-03-24T11:52:31.039 INFO:teuthology.orchestra.run.smithi090.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-24T11:52:31.040 INFO:teuthology.orchestra.run.smithi090.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-24T11:52:31.040 INFO:teuthology.orchestra.run.smithi090.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-24T11:52:31.040 INFO:teuthology.orchestra.run.smithi090.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-24T11:52:31.040 INFO:teuthology.orchestra.run.smithi090.stdout:Packager : builder@centos.org 2024-03-24T11:52:31.040 INFO:teuthology.orchestra.run.smithi090.stdout:Vendor : CentOS 2024-03-24T11:52:31.040 INFO:teuthology.orchestra.run.smithi090.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-24T11:52:31.040 INFO:teuthology.orchestra.run.smithi090.stdout:Summary : Support tools for GRUB. 2024-03-24T11:52:31.040 INFO:teuthology.orchestra.run.smithi090.stdout:Description : 2024-03-24T11:52:31.040 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T11:52:31.040 INFO:teuthology.orchestra.run.smithi090.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-24T11:52:31.040 INFO:teuthology.orchestra.run.smithi090.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-24T11:52:31.040 INFO:teuthology.orchestra.run.smithi090.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-24T11:52:31.040 INFO:teuthology.orchestra.run.smithi090.stdout:hardware devices. 2024-03-24T11:52:31.040 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T11:52:31.040 INFO:teuthology.orchestra.run.smithi090.stdout:This subpackage provides tools for support of all platforms. 2024-03-24T11:52:31.041 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-24T11:52:31.041 INFO:teuthology.task.kernel:Updating grub on smithi090 to boot 5.14.0-430.el9.x86_64 2024-03-24T11:52:31.041 DEBUG:teuthology.orchestra.run.smithi090:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-24T11:52:31.666 INFO:teuthology.orchestra.run.smithi090.stderr:Generating grub configuration file ... 2024-03-24T11:52:33.534 INFO:teuthology.orchestra.run.smithi090.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-24T11:52:33.554 INFO:teuthology.orchestra.run.smithi090.stderr:done 2024-03-24T11:52:33.556 DEBUG:teuthology.orchestra.run.smithi090:> mktemp 2024-03-24T11:52:33.588 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/tmp.a2rDD4Sg3A 2024-03-24T11:52:33.588 DEBUG:teuthology.orchestra.run.smithi090:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.a2rDD4Sg3A 2024-03-24T11:52:33.656 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0666 /tmp/tmp.a2rDD4Sg3A 2024-03-24T11:52:33.811 DEBUG:teuthology.orchestra.remote:smithi090:/tmp/tmp.a2rDD4Sg3A is 6KB 2024-03-24T11:52:33.831 DEBUG:teuthology.orchestra.run.smithi090:> rm -fr /tmp/tmp.a2rDD4Sg3A 2024-03-24T11:52:33.844 DEBUG:teuthology.orchestra.run.smithi090:> sudo /bin/ls /boot/loader/entries || true 2024-03-24T11:52:33.909 INFO:teuthology.orchestra.run.smithi090.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-24T11:52:33.909 INFO:teuthology.orchestra.run.smithi090.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-24T11:52:33.909 INFO:teuthology.orchestra.run.smithi090.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-24T11:52:33.909 INFO:teuthology.orchestra.run.smithi090.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-24T11:52:33.910 DEBUG:teuthology.orchestra.run.smithi090:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-24T11:52:33.983 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-24T11:52:34.027 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:52:34.061 DEBUG:teuthology.orchestra.run.smithi090:> sudo shutdown -r now 2024-03-24T11:52:39.129 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:53:04.064 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-24T11:53:04.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-24T11:53:04.065 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T11:53:10.038 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:53:10.038 INFO:teuthology.orchestra.run.smithi006.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-24T11:53:10.038 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T11:53:10.082 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:53:10.458 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:53:10.458 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-24T11:53:10.458 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-24T11:53:10.458 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-24T11:53:10.866 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-24T11:53:10.866 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T11:53:10.866 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-24T11:53:10.866 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-24T11:53:10.866 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-24T11:53:10.866 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-24T11:53:10.866 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-24T11:53:10.866 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T11:53:10.866 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T11:53:11.214 DEBUG:teuthology.orchestra.run.smithi006:> rpm -q kernel | sort -rV | head -n 1 2024-03-24T11:53:11.320 INFO:teuthology.orchestra.run.smithi006.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-24T11:53:11.321 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-24T11:53:11.322 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-24T11:53:11.322 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-24T11:53:11.322 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-24T11:53:11.322 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-24T11:53:11.322 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi006.front.sepia.ceph.com, path=None, version=distro) 2024-03-24T11:53:11.322 DEBUG:teuthology.orchestra.run.smithi006:> rpm -q kernel | sort -rV | head -n 1 2024-03-24T11:53:11.404 INFO:teuthology.orchestra.run.smithi006.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-24T11:53:11.404 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-24T11:53:11.404 DEBUG:teuthology.orchestra.run.smithi006:> sudo rpm -qi grub2-tools 2024-03-24T11:53:11.448 INFO:teuthology.orchestra.run.smithi006.stdout:Name : grub2-tools 2024-03-24T11:53:11.448 INFO:teuthology.orchestra.run.smithi006.stdout:Epoch : 1 2024-03-24T11:53:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:Version : 2.06 2024-03-24T11:53:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:Release : 68.el9 2024-03-24T11:53:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:Architecture: x86_64 2024-03-24T11:53:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-24T11:53:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:Group : Unspecified 2024-03-24T11:53:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:Size : 8264254 2024-03-24T11:53:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:License : GPLv3+ 2024-03-24T11:53:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-24T11:53:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-24T11:53:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-24T11:53:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-24T11:53:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:Packager : builder@centos.org 2024-03-24T11:53:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:Vendor : CentOS 2024-03-24T11:53:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-24T11:53:11.449 INFO:teuthology.orchestra.run.smithi006.stdout:Summary : Support tools for GRUB. 2024-03-24T11:53:11.450 INFO:teuthology.orchestra.run.smithi006.stdout:Description : 2024-03-24T11:53:11.450 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T11:53:11.450 INFO:teuthology.orchestra.run.smithi006.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-24T11:53:11.450 INFO:teuthology.orchestra.run.smithi006.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-24T11:53:11.450 INFO:teuthology.orchestra.run.smithi006.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-24T11:53:11.450 INFO:teuthology.orchestra.run.smithi006.stdout:hardware devices. 2024-03-24T11:53:11.450 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T11:53:11.450 INFO:teuthology.orchestra.run.smithi006.stdout:This subpackage provides tools for support of all platforms. 2024-03-24T11:53:11.451 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-24T11:53:11.451 INFO:teuthology.task.kernel:Updating grub on smithi006 to boot 5.14.0-430.el9.x86_64 2024-03-24T11:53:11.451 DEBUG:teuthology.orchestra.run.smithi006:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-24T11:53:12.017 INFO:teuthology.orchestra.run.smithi006.stderr:Generating grub configuration file ... 2024-03-24T11:53:13.366 INFO:teuthology.orchestra.run.smithi006.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-24T11:53:13.411 INFO:teuthology.orchestra.run.smithi006.stderr:done 2024-03-24T11:53:13.414 DEBUG:teuthology.orchestra.run.smithi006:> mktemp 2024-03-24T11:53:13.430 INFO:teuthology.orchestra.run.smithi006.stdout:/tmp/tmp.gFLdrRD3Q0 2024-03-24T11:53:13.431 DEBUG:teuthology.orchestra.run.smithi006:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.gFLdrRD3Q0 2024-03-24T11:53:13.505 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0666 /tmp/tmp.gFLdrRD3Q0 2024-03-24T11:53:13.651 DEBUG:teuthology.orchestra.remote:smithi006:/tmp/tmp.gFLdrRD3Q0 is 6KB 2024-03-24T11:53:13.700 DEBUG:teuthology.orchestra.run.smithi006:> rm -fr /tmp/tmp.gFLdrRD3Q0 2024-03-24T11:53:13.714 DEBUG:teuthology.orchestra.run.smithi006:> sudo /bin/ls /boot/loader/entries || true 2024-03-24T11:53:13.783 INFO:teuthology.orchestra.run.smithi006.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-24T11:53:13.783 INFO:teuthology.orchestra.run.smithi006.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-24T11:53:13.783 INFO:teuthology.orchestra.run.smithi006.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-24T11:53:13.783 INFO:teuthology.orchestra.run.smithi006.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-24T11:53:13.785 DEBUG:teuthology.orchestra.run.smithi006:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-24T11:53:13.934 DEBUG:teuthology.orchestra.run.smithi006:> sudo shutdown -r now 2024-03-24T11:53:22.462 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-03-24T11:53:31.470 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-24T11:53:31.471 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T11:53:34.526 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-03-24T11:53:43.944 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-24T11:53:43.944 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-03-24T11:53:43.945 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T11:53:46.529 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-24T11:53:46.530 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T11:53:49.598 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-03-24T11:54:02.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.6 2024-03-24T11:54:04.601 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-24T11:54:04.601 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T11:54:11.402 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-03-24T11:54:11.403 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T11:54:23.134 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-03-24T11:54:41.150 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-24T11:54:41.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T11:54:41.672 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-03-24T11:54:42.149 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-24T11:54:42.150 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-430.el9.x86_64"... 2024-03-24T11:54:42.150 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2024-03-24T11:54:42.164 INFO:teuthology.orchestra.run.smithi090.stdout:5.14.0-430.el9.x86_64 2024-03-24T11:54:42.164 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-24T11:54:42.164 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-24T11:54:42.164 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-24T11:54:43.165 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-24T11:54:43.166 DEBUG:teuthology.orchestra.run.smithi090:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-24T11:54:43.530 INFO:teuthology.orchestra.run.smithi090.stdout:ttyS1 2024-03-24T11:54:43.576 DEBUG:teuthology.parallel:result is None 2024-03-24T11:55:11.405 DEBUG:teuthology.orchestra.remote:timed out 2024-03-24T11:55:23.407 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-03-24T11:55:23.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T11:55:23.709 DEBUG:teuthology.orchestra.run.smithi006:> true 2024-03-24T11:55:24.117 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-03-24T11:55:24.118 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-430.el9.x86_64"... 2024-03-24T11:55:24.118 DEBUG:teuthology.orchestra.run.smithi006:> uname -r 2024-03-24T11:55:24.133 INFO:teuthology.orchestra.run.smithi006.stdout:5.14.0-430.el9.x86_64 2024-03-24T11:55:24.133 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-24T11:55:24.133 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-24T11:55:24.133 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-24T11:55:25.134 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-24T11:55:25.134 DEBUG:teuthology.orchestra.run.smithi006:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-24T11:55:25.218 INFO:teuthology.orchestra.run.smithi006.stdout:ttyS1 2024-03-24T11:55:25.251 DEBUG:teuthology.parallel:result is None 2024-03-24T11:55:25.252 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-24T11:55:25.259 INFO:teuthology.task.internal:Creating test directory... 2024-03-24T11:55:25.259 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-24T11:55:25.262 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-24T11:55:25.283 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-24T11:55:25.289 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-24T11:55:25.300 INFO:teuthology.task.internal:Creating archive directory... 2024-03-24T11:55:25.301 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-24T11:55:25.321 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-24T11:55:25.350 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-24T11:55:25.357 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-24T11:55:25.357 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-24T11:55:25.382 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-24T11:55:25.422 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-24T11:55:25.431 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-24T11:55:25.436 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-24T11:55:25.444 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-24T11:55:25.446 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-24T11:55:25.453 INFO:teuthology.task.internal:Configuring sudo... 2024-03-24T11:55:25.453 DEBUG:teuthology.orchestra.run.smithi006:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-24T11:55:25.456 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-24T11:55:25.507 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-24T11:55:25.516 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-24T11:55:25.517 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-24T11:55:25.525 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-24T11:55:25.564 DEBUG:teuthology.orchestra.run.smithi006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-24T11:55:25.620 DEBUG:teuthology.orchestra.run.smithi006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-24T11:55:25.681 DEBUG:teuthology.orchestra.run.smithi006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-24T11:55:25.763 DEBUG:teuthology.orchestra.run.smithi006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-24T11:55:25.808 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-24T11:55:25.808 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-24T11:55:25.876 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-24T11:55:25.919 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-24T11:55:25.975 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-24T11:55:26.055 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-24T11:55:26.103 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-24T11:55:26.103 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-24T11:55:26.171 DEBUG:teuthology.orchestra.run.smithi006:> sudo service rsyslog restart 2024-03-24T11:55:26.174 DEBUG:teuthology.orchestra.run.smithi090:> sudo service rsyslog restart 2024-03-24T11:55:26.222 INFO:teuthology.orchestra.run.smithi006.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-24T11:55:26.267 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-24T11:55:26.719 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-24T11:55:26.726 INFO:teuthology.task.internal:Starting timer... 2024-03-24T11:55:26.726 INFO:teuthology.run_tasks:Running task pcp... 2024-03-24T11:55:26.739 INFO:teuthology.run_tasks:Running task selinux... 2024-03-24T11:55:26.747 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-24T11:55:26.748 DEBUG:teuthology.orchestra.run.smithi006:> sudo service auditd rotate 2024-03-24T11:55:26.799 INFO:teuthology.orchestra.run.smithi006.stdout:Rotating logs: 2024-03-24T11:55:26.801 DEBUG:teuthology.orchestra.run.smithi090:> sudo service auditd rotate 2024-03-24T11:55:26.862 INFO:teuthology.orchestra.run.smithi090.stdout:Rotating logs: 2024-03-24T11:55:26.864 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-24T11:55:26.864 DEBUG:teuthology.orchestra.run.smithi006:> /usr/sbin/getenforce 2024-03-24T11:55:26.889 INFO:teuthology.orchestra.run.smithi006.stdout:Permissive 2024-03-24T11:55:26.889 DEBUG:teuthology.orchestra.run.smithi090:> /usr/sbin/getenforce 2024-03-24T11:55:26.923 INFO:teuthology.orchestra.run.smithi090.stdout:Permissive 2024-03-24T11:55:26.923 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi006.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi090.front.sepia.ceph.com': 'permissive'} 2024-03-24T11:55:26.924 DEBUG:teuthology.orchestra.run.smithi006:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-24T11:55:26.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T11:55:26.958 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-24T11:55:26.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T11:55:26.992 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-24T11:55:26.992 DEBUG:teuthology.orchestra.run.smithi006:> sudo /usr/sbin/setenforce permissive 2024-03-24T11:55:27.040 DEBUG:teuthology.orchestra.run.smithi090:> sudo /usr/sbin/setenforce permissive 2024-03-24T11:55:27.073 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-24T11:55:27.082 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-24T11:55:27.099 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-24T11:55:27.100 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi006.front.sepia.ceph.com,smithi090.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-24T11:59:47.133 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi006.front.sepia.ceph.com'), Remote(name='ubuntu@smithi090.front.sepia.ceph.com')] 2024-03-24T11:59:47.134 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-03-24T11:59:47.135 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T11:59:47.225 DEBUG:teuthology.orchestra.run.smithi006:> true 2024-03-24T11:59:47.308 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com' 2024-03-24T11:59:47.309 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-24T11:59:47.309 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-24T11:59:47.399 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-03-24T11:59:47.482 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-24T11:59:47.483 INFO:teuthology.run_tasks:Running task clock... 2024-03-24T11:59:47.495 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-24T11:59:47.495 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-24T11:59:47.496 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-24T11:59:47.498 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-24T11:59:47.498 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-24T11:59:47.526 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-24T11:59:47.542 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-24T11:59:47.555 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-24T11:59:47.572 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-24T11:59:47.572 INFO:teuthology.orchestra.run.smithi006.stderr:sudo: ntpd: command not found 2024-03-24T11:59:47.586 INFO:teuthology.orchestra.run.smithi006.stdout:506 Cannot talk to daemon 2024-03-24T11:59:47.601 INFO:teuthology.orchestra.run.smithi090.stderr:sudo: ntpd: command not found 2024-03-24T11:59:47.604 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-24T11:59:47.615 INFO:teuthology.orchestra.run.smithi090.stdout:506 Cannot talk to daemon 2024-03-24T11:59:47.622 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-24T11:59:47.631 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-24T11:59:47.647 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-24T11:59:47.683 INFO:teuthology.orchestra.run.smithi006.stderr:bash: line 1: ntpq: command not found 2024-03-24T11:59:47.686 INFO:teuthology.orchestra.run.smithi006.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-24T11:59:47.686 INFO:teuthology.orchestra.run.smithi006.stdout:=============================================================================== 2024-03-24T11:59:47.686 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T11:59:47.686 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T11:59:47.686 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T11:59:47.687 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T11:59:47.698 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-03-24T11:59:47.702 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-24T11:59:47.702 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-03-24T11:59:47.702 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T11:59:47.702 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T11:59:47.702 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T11:59:47.702 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T11:59:47.703 INFO:teuthology.run_tasks:Running task pexec... 2024-03-24T11:59:47.712 INFO:teuthology.task.pexec:Executing custom commands... 2024-03-24T11:59:47.713 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi006.front.sepia.ceph.com 2024-03-24T11:59:47.713 DEBUG:teuthology.orchestra.run.smithi006:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-24T11:59:47.713 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi090.front.sepia.ceph.com 2024-03-24T11:59:47.714 DEBUG:teuthology.orchestra.run.smithi090:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-24T11:59:48.317 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:00:50 ago on Sun 24 Mar 2024 11:58:58 AM UTC. 2024-03-24T11:59:48.326 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:49 ago on Sun 24 Mar 2024 11:58:59 AM UTC. 2024-03-24T11:59:48.469 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T11:59:48.471 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T11:59:48.471 INFO:teuthology.orchestra.run.smithi006.stdout: Package Architecture Version Repository Size 2024-03-24T11:59:48.471 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T11:59:48.471 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-24T11:59:48.471 INFO:teuthology.orchestra.run.smithi006.stdout: runc x86_64 4:1.1.12-2.el9 appstream 3.1 M 2024-03-24T11:59:48.471 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T11:59:48.471 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-24T11:59:48.471 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T11:59:48.472 INFO:teuthology.orchestra.run.smithi006.stdout:Install 1 Package 2024-03-24T11:59:48.472 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T11:59:48.472 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T11:59:48.480 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 3.1 M 2024-03-24T11:59:48.481 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 9.8 M 2024-03-24T11:59:48.481 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-24T11:59:48.513 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T11:59:48.513 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-03-24T11:59:48.513 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T11:59:48.513 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-03-24T11:59:48.513 INFO:teuthology.orchestra.run.smithi090.stdout: runc x86_64 4:1.1.12-2.el9 appstream 3.1 M 2024-03-24T11:59:48.513 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T11:59:48.513 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-03-24T11:59:48.513 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T11:59:48.513 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-03-24T11:59:48.513 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T11:59:48.521 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 3.1 M 2024-03-24T11:59:48.521 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 9.8 M 2024-03-24T11:59:48.521 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-03-24T11:59:49.275 INFO:teuthology.orchestra.run.smithi090.stdout:runc-1.1.12-2.el9.x86_64.rpm 5.2 MB/s | 3.1 MB 00:00 2024-03-24T11:59:49.275 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-03-24T11:59:49.275 INFO:teuthology.orchestra.run.smithi090.stdout:Total 4.1 MB/s | 3.1 MB 00:00 2024-03-24T11:59:49.312 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-03-24T11:59:49.318 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-03-24T11:59:49.318 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-03-24T11:59:49.398 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-03-24T11:59:49.398 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-03-24T11:59:49.454 INFO:teuthology.orchestra.run.smithi006.stdout:runc-1.1.12-2.el9.x86_64.rpm 3.4 MB/s | 3.1 MB 00:00 2024-03-24T11:59:49.454 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-24T11:59:49.454 INFO:teuthology.orchestra.run.smithi006.stdout:Total 3.2 MB/s | 3.1 MB 00:00 2024-03-24T11:59:49.491 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-24T11:59:49.497 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-24T11:59:49.497 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-24T11:59:49.581 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-24T11:59:49.581 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-24T11:59:49.639 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-03-24T11:59:49.746 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-24T11:59:49.814 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-24T11:59:49.909 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-24T11:59:50.434 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-24T11:59:50.479 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-24T11:59:50.765 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-24T11:59:50.766 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T11:59:50.766 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-24T11:59:50.766 INFO:teuthology.orchestra.run.smithi006.stdout: runc-4:1.1.12-2.el9.x86_64 2024-03-24T11:59:50.766 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T11:59:50.766 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T11:59:50.776 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-24T11:59:50.776 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T11:59:50.776 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-03-24T11:59:50.776 INFO:teuthology.orchestra.run.smithi090.stdout: runc-4:1.1.12-2.el9.x86_64 2024-03-24T11:59:50.776 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T11:59:50.776 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T11:59:50.995 DEBUG:teuthology.parallel:result is None 2024-03-24T11:59:51.104 DEBUG:teuthology.parallel:result is None 2024-03-24T11:59:51.104 INFO:teuthology.run_tasks:Running task install... 2024-03-24T11:59:51.117 DEBUG:teuthology.task.install:project ceph 2024-03-24T11:59:51.117 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'db0330b1e4e2470d52b750e251e55a522b4f7d69'}} 2024-03-24T11:59:51.117 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'db0330b1e4e2470d52b750e251e55a522b4f7d69'} 2024-03-24T11:59:51.117 INFO:teuthology.task.install:Using flavor: default 2024-03-24T11:59:51.127 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-24T11:59:51.127 INFO:teuthology.task.install:extra packages: [] 2024-03-24T11:59:51.128 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'db0330b1e4e2470d52b750e251e55a522b4f7d69', 'tag': None, 'wait_for_package': False} 2024-03-24T11:59:51.128 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:59:51.130 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'db0330b1e4e2470d52b750e251e55a522b4f7d69', 'tag': None, 'wait_for_package': False} 2024-03-24T11:59:51.130 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T11:59:51.270 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/db0330b1e4e2470d52b750e251e55a522b4f7d69/centos/9/flavors/default/ 2024-03-24T11:59:51.270 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1667.gdb0330b1 2024-03-24T11:59:51.356 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/db0330b1e4e2470d52b750e251e55a522b4f7d69/centos/9/flavors/default/ 2024-03-24T11:59:51.357 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1667.gdb0330b1 2024-03-24T11:59:51.379 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/db0330b1e4e2470d52b750e251e55a522b4f7d69/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/db0330b1e4e2470d52b750e251e55a522b4f7d69/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/db0330b1e4e2470d52b750e251e55a522b4f7d69/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-24T11:59:51.381 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-24T11:59:51.381 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-24T11:59:51.418 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-24T11:59:51.419 DEBUG:teuthology.orchestra.run.smithi006:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/db0330b1e4e2470d52b750e251e55a522b4f7d69/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-24T11:59:51.467 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/db0330b1e4e2470d52b750e251e55a522b4f7d69/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/db0330b1e4e2470d52b750e251e55a522b4f7d69/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/db0330b1e4e2470d52b750e251e55a522b4f7d69/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-24T11:59:51.468 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-24T11:59:51.468 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-24T11:59:51.487 DEBUG:teuthology.orchestra.run.smithi006:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-24T11:59:51.494 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-24T11:59:51.494 DEBUG:teuthology.orchestra.run.smithi090:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/db0330b1e4e2470d52b750e251e55a522b4f7d69/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-24T11:59:51.561 DEBUG:teuthology.orchestra.run.smithi090:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-24T11:59:51.568 DEBUG:teuthology.orchestra.run.smithi006:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-24T11:59:51.635 INFO:teuthology.orchestra.run.smithi006.stdout:check_obsoletes = 1 2024-03-24T11:59:51.636 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum clean all 2024-03-24T11:59:51.676 DEBUG:teuthology.orchestra.run.smithi090:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-24T11:59:51.702 INFO:teuthology.orchestra.run.smithi090.stdout:check_obsoletes = 1 2024-03-24T11:59:51.703 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum clean all 2024-03-24T11:59:51.879 INFO:teuthology.orchestra.run.smithi006.stdout:74 files removed 2024-03-24T11:59:51.906 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-radosgw 2024-03-24T11:59:51.960 INFO:teuthology.orchestra.run.smithi090.stdout:74 files removed 2024-03-24T11:59:51.987 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-radosgw 2024-03-24T11:59:52.703 INFO:teuthology.orchestra.run.smithi090.stdout:ceph packages for x86_64 196 kB/s | 82 kB 00:00 2024-03-24T11:59:52.712 INFO:teuthology.orchestra.run.smithi006.stdout:ceph packages for x86_64 163 kB/s | 82 kB 00:00 2024-03-24T11:59:53.262 INFO:teuthology.orchestra.run.smithi090.stdout:ceph noarch packages 20 kB/s | 10 kB 00:00 2024-03-24T11:59:53.404 INFO:teuthology.orchestra.run.smithi006.stdout:ceph noarch packages 16 kB/s | 10 kB 00:00 2024-03-24T11:59:53.737 INFO:teuthology.orchestra.run.smithi090.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2024-03-24T11:59:53.821 INFO:teuthology.orchestra.run.smithi006.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-03-24T11:59:54.349 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-24T11:59:54.482 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-24T11:59:56.058 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS-9 - BaseOS 3.4 MB/s | 2.0 MB 00:00 2024-03-24T11:59:56.132 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-9 - BaseOS 4.3 MB/s | 2.0 MB 00:00 2024-03-24T11:59:59.281 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.0 MB 00:02 2024-03-24T12:00:01.241 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-03-24T12:00:05.565 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 896 kB/s | 8.0 MB 00:09 2024-03-24T12:00:06.342 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.1 MB 00:01 2024-03-24T12:00:07.751 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 16 kB 00:00 2024-03-24T12:00:09.847 INFO:teuthology.orchestra.run.smithi006.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:01 2024-03-24T12:00:14.978 INFO:teuthology.orchestra.run.smithi006.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-24T12:00:17.527 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:00:17.529 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:00:17.529 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-03-24T12:00:17.529 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:00:17.529 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-24T12:00:17.529 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-radosgw x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 9.8 M 2024-03-24T12:00:17.529 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrading: 2024-03-24T12:00:17.530 INFO:teuthology.orchestra.run.smithi006.stdout: librados2 x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 3.4 M 2024-03-24T12:00:17.530 INFO:teuthology.orchestra.run.smithi006.stdout: librbd1 x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 3.1 M 2024-03-24T12:00:17.530 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-03-24T12:00:17.530 INFO:teuthology.orchestra.run.smithi006.stdout: boost-program-options 2024-03-24T12:00:17.530 INFO:teuthology.orchestra.run.smithi006.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-24T12:00:17.530 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-base x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 5.3 M 2024-03-24T12:00:17.530 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-common x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 20 M 2024-03-24T12:00:17.530 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-selinux x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 25 k 2024-03-24T12:00:17.530 INFO:teuthology.orchestra.run.smithi006.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-24T12:00:17.530 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-24T12:00:17.530 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-24T12:00:17.530 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs2 x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 742 k 2024-03-24T12:00:17.530 INFO:teuthology.orchestra.run.smithi006.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-24T12:00:17.530 INFO:teuthology.orchestra.run.smithi006.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-24T12:00:17.531 INFO:teuthology.orchestra.run.smithi006.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-24T12:00:17.531 INFO:teuthology.orchestra.run.smithi006.stdout: libradosstriper1 x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 498 k 2024-03-24T12:00:17.531 INFO:teuthology.orchestra.run.smithi006.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-24T12:00:17.531 INFO:teuthology.orchestra.run.smithi006.stdout: librgw2 x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 5.1 M 2024-03-24T12:00:17.531 INFO:teuthology.orchestra.run.smithi006.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-24T12:00:17.531 INFO:teuthology.orchestra.run.smithi006.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-24T12:00:17.531 INFO:teuthology.orchestra.run.smithi006.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-24T12:00:17.531 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-argparse 2024-03-24T12:00:17.531 INFO:teuthology.orchestra.run.smithi006.stdout: x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 45 k 2024-03-24T12:00:17.531 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-common x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 131 k 2024-03-24T12:00:17.531 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cephfs x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 161 k 2024-03-24T12:00:17.531 INFO:teuthology.orchestra.run.smithi006.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-24T12:00:17.532 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rados x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 320 k 2024-03-24T12:00:17.532 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rbd x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 298 k 2024-03-24T12:00:17.532 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rgw x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 100 k 2024-03-24T12:00:17.532 INFO:teuthology.orchestra.run.smithi006.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-24T12:00:17.532 INFO:teuthology.orchestra.run.smithi006.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-24T12:00:17.532 INFO:teuthology.orchestra.run.smithi006.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-24T12:00:17.532 INFO:teuthology.orchestra.run.smithi006.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-24T12:00:17.532 INFO:teuthology.orchestra.run.smithi006.stdout:Installing weak dependencies: 2024-03-24T12:00:17.532 INFO:teuthology.orchestra.run.smithi006.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-24T12:00:17.532 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:00:17.532 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-24T12:00:17.532 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:00:17.532 INFO:teuthology.orchestra.run.smithi006.stdout:Install 30 Packages 2024-03-24T12:00:17.533 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrade 2 Packages 2024-03-24T12:00:17.533 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:00:17.533 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 58 M 2024-03-24T12:00:17.533 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-24T12:00:18.412 INFO:teuthology.orchestra.run.smithi006.stdout:(1/32): ceph-base-19.0.0-1667.gdb0330b1.el9.x86 8.0 MB/s | 5.3 MB 00:00 2024-03-24T12:00:18.454 INFO:teuthology.orchestra.run.smithi006.stdout:(2/32): ceph-selinux-19.0.0-1667.gdb0330b1.el9. 608 kB/s | 25 kB 00:00 2024-03-24T12:00:18.538 INFO:teuthology.orchestra.run.smithi006.stdout:(3/32): libcephfs2-19.0.0-1667.gdb0330b1.el9.x8 8.7 MB/s | 742 kB 00:00 2024-03-24T12:00:18.604 INFO:teuthology.orchestra.run.smithi006.stdout:(4/32): libradosstriper1-19.0.0-1667.gdb0330b1. 7.3 MB/s | 498 kB 00:00 2024-03-24T12:00:18.955 INFO:teuthology.orchestra.run.smithi006.stdout:(5/32): librgw2-19.0.0-1667.gdb0330b1.el9.x86_6 15 MB/s | 5.1 MB 00:00 2024-03-24T12:00:18.988 INFO:teuthology.orchestra.run.smithi006.stdout:(6/32): python3-ceph-argparse-19.0.0-1667.gdb03 1.3 MB/s | 45 kB 00:00 2024-03-24T12:00:19.039 INFO:teuthology.orchestra.run.smithi006.stdout:(7/32): python3-ceph-common-19.0.0-1667.gdb0330 2.6 MB/s | 131 kB 00:00 2024-03-24T12:00:19.089 INFO:teuthology.orchestra.run.smithi006.stdout:(8/32): python3-cephfs-19.0.0-1667.gdb0330b1.el 3.1 MB/s | 161 kB 00:00 2024-03-24T12:00:19.147 INFO:teuthology.orchestra.run.smithi006.stdout:(9/32): python3-rados-19.0.0-1667.gdb0330b1.el9 5.4 MB/s | 320 kB 00:00 2024-03-24T12:00:19.206 INFO:teuthology.orchestra.run.smithi006.stdout:(10/32): python3-rbd-19.0.0-1667.gdb0330b1.el9. 5.0 MB/s | 298 kB 00:00 2024-03-24T12:00:19.256 INFO:teuthology.orchestra.run.smithi006.stdout:(11/32): python3-rgw-19.0.0-1667.gdb0330b1.el9. 1.9 MB/s | 100 kB 00:00 2024-03-24T12:00:19.457 INFO:teuthology.orchestra.run.smithi006.stdout:(12/32): boost-program-options-1.75.0-8.el9.x86 529 kB/s | 106 kB 00:00 2024-03-24T12:00:19.532 INFO:teuthology.orchestra.run.smithi006.stdout:(13/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-03-24T12:00:19.557 INFO:teuthology.orchestra.run.smithi006.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.7 MB/s | 44 kB 00:00 2024-03-24T12:00:19.632 INFO:teuthology.orchestra.run.smithi006.stdout:(15/32): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-24T12:00:19.758 INFO:teuthology.orchestra.run.smithi006.stdout:(16/32): ceph-radosgw-19.0.0-1667.gdb0330b1.el9 4.8 MB/s | 9.8 MB 00:02 2024-03-24T12:00:19.783 INFO:teuthology.orchestra.run.smithi006.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.8 MB/s | 282 kB 00:00 2024-03-24T12:00:19.800 INFO:teuthology.orchestra.run.smithi006.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 998 kB/s | 41 kB 00:00 2024-03-24T12:00:19.984 INFO:teuthology.orchestra.run.smithi006.stdout:(19/32): qatlib-service-23.11.0-1.el9.x86_64.rp 190 kB/s | 35 kB 00:00 2024-03-24T12:00:20.026 INFO:teuthology.orchestra.run.smithi006.stdout:(20/32): qatlib-23.11.0-1.el9.x86_64.rpm 894 kB/s | 216 kB 00:00 2024-03-24T12:00:20.059 INFO:teuthology.orchestra.run.smithi006.stdout:(21/32): qatzip-libs-1.1.2-2.el9.x86_64.rpm 592 kB/s | 44 kB 00:00 2024-03-24T12:00:20.343 INFO:teuthology.orchestra.run.smithi006.stdout:(22/32): ceph-common-19.0.0-1667.gdb0330b1.el9. 7.7 MB/s | 20 MB 00:02 2024-03-24T12:00:20.610 INFO:teuthology.orchestra.run.smithi006.stdout:(23/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 529 kB/s | 309 kB 00:00 2024-03-24T12:00:20.635 INFO:teuthology.orchestra.run.smithi006.stdout:(24/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 89 kB/s | 26 kB 00:00 2024-03-24T12:00:20.897 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:14 2024-03-24T12:00:20.952 INFO:teuthology.orchestra.run.smithi006.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 142 kB/s | 49 kB 00:00 2024-03-24T12:00:20.977 INFO:teuthology.orchestra.run.smithi006.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 197 kB/s | 67 kB 00:00 2024-03-24T12:00:21.094 INFO:teuthology.orchestra.run.smithi006.stdout:(27/32): libarrow-9.0.0-10.el9.x86_64.rpm 4.3 MB/s | 4.4 MB 00:01 2024-03-24T12:00:21.436 INFO:teuthology.orchestra.run.smithi006.stdout:(28/32): re2-20211101-3.el9.x86_64.rpm 418 kB/s | 192 kB 00:00 2024-03-24T12:00:21.636 INFO:teuthology.orchestra.run.smithi006.stdout:(29/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.2 MB/s | 838 kB 00:00 2024-03-24T12:00:21.862 INFO:teuthology.orchestra.run.smithi006.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2024-03-24T12:00:21.962 INFO:teuthology.orchestra.run.smithi006.stdout:(31/32): librados2-19.0.0-1667.gdb0330b1.el9.x8 6.4 MB/s | 3.4 MB 00:00 2024-03-24T12:00:22.429 INFO:teuthology.orchestra.run.smithi006.stdout:(32/32): librbd1-19.0.0-1667.gdb0330b1.el9.x86_ 4.0 MB/s | 3.1 MB 00:00 2024-03-24T12:00:22.435 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:00:22.435 INFO:teuthology.orchestra.run.smithi006.stdout:Total 12 MB/s | 58 MB 00:04 2024-03-24T12:00:22.469 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-24T12:00:22.510 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-24T12:00:22.510 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-24T12:00:23.012 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-24T12:00:23.012 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-24T12:00:24.460 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-24T12:00:24.542 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-03-24T12:00:24.575 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-03-24T12:00:24.618 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-03-24T12:00:24.648 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-03-24T12:00:24.664 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-03-24T12:00:24.818 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 5/34 2024-03-24T12:00:24.871 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 5/34 2024-03-24T12:00:25.219 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-03-24T12:00:25.269 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : librados2-2:19.0.0-1667.gdb0330b1.el9.x86_64 7/34 2024-03-24T12:00:25.345 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librados2-2:19.0.0-1667.gdb0330b1.el9.x86_64 7/34 2024-03-24T12:00:25.434 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rados-2:19.0.0-1667.gdb0330b1.el9.x86_64 8/34 2024-03-24T12:00:25.462 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephfs2-2:19.0.0-1667.gdb0330b1.el9.x86_64 9/34 2024-03-24T12:00:25.525 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libcephfs2-2:19.0.0-1667.gdb0330b1.el9.x86_64 9/34 2024-03-24T12:00:25.554 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-03-24T12:00:25.598 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ceph-argparse-2:19.0.0-1667.gdb0330b1.el9. 11/34 2024-03-24T12:00:25.667 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cephfs-2:19.0.0-1667.gdb0330b1.el9.x86_64 12/34 2024-03-24T12:00:25.689 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libradosstriper1-2:19.0.0-1667.gdb0330b1.el9.x86_6 13/34 2024-03-24T12:00:25.737 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1667.gdb0330b1.el9.x86_6 13/34 2024-03-24T12:00:25.763 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 14/34 2024-03-24T12:00:25.875 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-03-24T12:00:25.914 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-03-24T12:00:25.955 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-03-24T12:00:25.988 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-03-24T12:00:26.017 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-03-24T12:00:26.049 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-03-24T12:00:26.376 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-03-24T12:00:26.404 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : librbd1-2:19.0.0-1667.gdb0330b1.el9.x86_64 22/34 2024-03-24T12:00:26.478 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librbd1-2:19.0.0-1667.gdb0330b1.el9.x86_64 22/34 2024-03-24T12:00:26.509 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rbd-2:19.0.0-1667.gdb0330b1.el9.x86_64 23/34 2024-03-24T12:00:26.550 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-03-24T12:00:27.033 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-03-24T12:00:27.056 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librgw2-2:19.0.0-1667.gdb0330b1.el9.x86_64 26/34 2024-03-24T12:00:27.128 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librgw2-2:19.0.0-1667.gdb0330b1.el9.x86_64 26/34 2024-03-24T12:00:27.198 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rgw-2:19.0.0-1667.gdb0330b1.el9.x86_64 27/34 2024-03-24T12:00:27.226 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ceph-common-2:19.0.0-1667.gdb0330b1.el9.x8 28/34 2024-03-24T12:00:29.447 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-common-2:19.0.0-1667.gdb0330b1.el9.x86_64 29/34 2024-03-24T12:00:29.480 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-common-2:19.0.0-1667.gdb0330b1.el9.x86_64 29/34 2024-03-24T12:00:29.903 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 1.1 MB/s | 6.1 MB 00:05 2024-03-24T12:00:30.069 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-common-2:19.0.0-1667.gdb0330b1.el9.x86_64 29/34 2024-03-24T12:00:30.101 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-base-2:19.0.0-1667.gdb0330b1.el9.x86_64 30/34 2024-03-24T12:00:30.177 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-base-2:19.0.0-1667.gdb0330b1.el9.x86_64 30/34 2024-03-24T12:00:30.177 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-24T12:00:30.177 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-24T12:00:30.177 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:00:30.218 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-selinux-2:19.0.0-1667.gdb0330b1.el9.x86_64 31/34 2024-03-24T12:00:32.362 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 16 kB 00:01 2024-03-24T12:00:33.956 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-03-24T12:00:38.883 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-03-24T12:00:40.163 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1667.gdb0330b1.el9.x86_64 31/34 2024-03-24T12:00:40.163 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /sys 2024-03-24T12:00:40.163 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /proc 2024-03-24T12:00:40.163 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /mnt 2024-03-24T12:00:40.163 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /var/tmp 2024-03-24T12:00:40.163 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /home 2024-03-24T12:00:40.163 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /root 2024-03-24T12:00:40.163 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /tmp 2024-03-24T12:00:40.163 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:00:41.096 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-radosgw-2:19.0.0-1667.gdb0330b1.el9.x86_64 32/34 2024-03-24T12:00:41.120 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1667.gdb0330b1.el9.x86_64 32/34 2024-03-24T12:00:41.120 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T12:00:41.120 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-24T12:00:41.120 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-24T12:00:41.121 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-24T12:00:41.121 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:00:41.122 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-24T12:00:41.190 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-24T12:00:41.190 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-24T12:00:41.324 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:00:41.326 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:00:41.327 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-03-24T12:00:41.327 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:00:41.327 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-03-24T12:00:41.327 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 9.8 M 2024-03-24T12:00:41.327 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrading: 2024-03-24T12:00:41.327 INFO:teuthology.orchestra.run.smithi090.stdout: librados2 x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 3.4 M 2024-03-24T12:00:41.327 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1 x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 3.1 M 2024-03-24T12:00:41.328 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-03-24T12:00:41.328 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options 2024-03-24T12:00:41.328 INFO:teuthology.orchestra.run.smithi090.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-24T12:00:41.328 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 5.3 M 2024-03-24T12:00:41.328 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 20 M 2024-03-24T12:00:41.328 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 25 k 2024-03-24T12:00:41.328 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-24T12:00:41.328 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-24T12:00:41.328 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-24T12:00:41.328 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2 x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 742 k 2024-03-24T12:00:41.329 INFO:teuthology.orchestra.run.smithi090.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-24T12:00:41.329 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-24T12:00:41.329 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-24T12:00:41.329 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1 x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 498 k 2024-03-24T12:00:41.329 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-24T12:00:41.329 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2 x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 5.1 M 2024-03-24T12:00:41.329 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-24T12:00:41.330 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-24T12:00:41.330 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-24T12:00:41.330 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse 2024-03-24T12:00:41.330 INFO:teuthology.orchestra.run.smithi090.stdout: x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 45 k 2024-03-24T12:00:41.330 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 131 k 2024-03-24T12:00:41.330 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 161 k 2024-03-24T12:00:41.330 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-24T12:00:41.330 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 320 k 2024-03-24T12:00:41.331 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 298 k 2024-03-24T12:00:41.331 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 100 k 2024-03-24T12:00:41.331 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-24T12:00:41.331 INFO:teuthology.orchestra.run.smithi090.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-24T12:00:41.331 INFO:teuthology.orchestra.run.smithi090.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-24T12:00:41.331 INFO:teuthology.orchestra.run.smithi090.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-24T12:00:41.331 INFO:teuthology.orchestra.run.smithi090.stdout:Installing weak dependencies: 2024-03-24T12:00:41.331 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-24T12:00:41.332 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:00:41.332 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-03-24T12:00:41.332 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:00:41.332 INFO:teuthology.orchestra.run.smithi090.stdout:Install 30 Packages 2024-03-24T12:00:41.332 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrade 2 Packages 2024-03-24T12:00:41.332 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:00:41.333 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 58 M 2024-03-24T12:00:41.333 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-03-24T12:00:42.068 INFO:teuthology.orchestra.run.smithi090.stdout:(1/32): ceph-base-19.0.0-1667.gdb0330b1.el9.x86 9.2 MB/s | 5.3 MB 00:00 2024-03-24T12:00:42.119 INFO:teuthology.orchestra.run.smithi090.stdout:(2/32): ceph-selinux-19.0.0-1667.gdb0330b1.el9. 504 kB/s | 25 kB 00:00 2024-03-24T12:00:42.202 INFO:teuthology.orchestra.run.smithi090.stdout:(3/32): libcephfs2-19.0.0-1667.gdb0330b1.el9.x8 8.7 MB/s | 742 kB 00:00 2024-03-24T12:00:42.250 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-24T12:00:42.250 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-base-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/34 2024-03-24T12:00:42.250 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-common-2:19.0.0-1667.gdb0330b1.el9.x86_64 2/34 2024-03-24T12:00:42.250 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-radosgw-2:19.0.0-1667.gdb0330b1.el9.x86_64 3/34 2024-03-24T12:00:42.251 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-selinux-2:19.0.0-1667.gdb0330b1.el9.x86_64 4/34 2024-03-24T12:00:42.251 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephfs2-2:19.0.0-1667.gdb0330b1.el9.x86_64 5/34 2024-03-24T12:00:42.251 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libradosstriper1-2:19.0.0-1667.gdb0330b1.el9.x86_6 6/34 2024-03-24T12:00:42.251 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librgw2-2:19.0.0-1667.gdb0330b1.el9.x86_64 7/34 2024-03-24T12:00:42.251 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1667.gdb0330b1.el9. 8/34 2024-03-24T12:00:42.251 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ceph-common-2:19.0.0-1667.gdb0330b1.el9.x8 9/34 2024-03-24T12:00:42.251 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cephfs-2:19.0.0-1667.gdb0330b1.el9.x86_64 10/34 2024-03-24T12:00:42.251 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rados-2:19.0.0-1667.gdb0330b1.el9.x86_64 11/34 2024-03-24T12:00:42.251 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rbd-2:19.0.0-1667.gdb0330b1.el9.x86_64 12/34 2024-03-24T12:00:42.251 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rgw-2:19.0.0-1667.gdb0330b1.el9.x86_64 13/34 2024-03-24T12:00:42.251 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-03-24T12:00:42.251 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-03-24T12:00:42.252 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-03-24T12:00:42.252 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-03-24T12:00:42.252 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-03-24T12:00:42.252 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-03-24T12:00:42.252 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 20/34 2024-03-24T12:00:42.252 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 21/34 2024-03-24T12:00:42.252 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 22/34 2024-03-24T12:00:42.252 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-03-24T12:00:42.252 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-03-24T12:00:42.252 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-03-24T12:00:42.252 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-03-24T12:00:42.253 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-03-24T12:00:42.253 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-03-24T12:00:42.253 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-03-24T12:00:42.253 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-03-24T12:00:42.253 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados2-2:19.0.0-1667.gdb0330b1.el9.x86_64 31/34 2024-03-24T12:00:42.253 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-03-24T12:00:42.253 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librbd1-2:19.0.0-1667.gdb0330b1.el9.x86_64 33/34 2024-03-24T12:00:42.260 INFO:teuthology.orchestra.run.smithi090.stdout:(4/32): libradosstriper1-19.0.0-1667.gdb0330b1. 8.3 MB/s | 498 kB 00:00 2024-03-24T12:00:42.594 INFO:teuthology.orchestra.run.smithi090.stdout:(5/32): librgw2-19.0.0-1667.gdb0330b1.el9.x86_6 15 MB/s | 5.1 MB 00:00 2024-03-24T12:00:42.644 INFO:teuthology.orchestra.run.smithi090.stdout:(6/32): python3-ceph-argparse-19.0.0-1667.gdb03 908 kB/s | 45 kB 00:00 2024-03-24T12:00:42.703 INFO:teuthology.orchestra.run.smithi090.stdout:(7/32): python3-ceph-common-19.0.0-1667.gdb0330 2.2 MB/s | 131 kB 00:00 2024-03-24T12:00:42.761 INFO:teuthology.orchestra.run.smithi090.stdout:(8/32): python3-cephfs-19.0.0-1667.gdb0330b1.el 2.7 MB/s | 161 kB 00:00 2024-03-24T12:00:42.837 INFO:teuthology.orchestra.run.smithi090.stdout:(9/32): python3-rados-19.0.0-1667.gdb0330b1.el9 4.2 MB/s | 320 kB 00:00 2024-03-24T12:00:42.912 INFO:teuthology.orchestra.run.smithi090.stdout:(10/32): python3-rbd-19.0.0-1667.gdb0330b1.el9. 3.9 MB/s | 298 kB 00:00 2024-03-24T12:00:42.979 INFO:teuthology.orchestra.run.smithi090.stdout:(11/32): python3-rgw-19.0.0-1667.gdb0330b1.el9. 1.5 MB/s | 100 kB 00:00 2024-03-24T12:00:43.080 INFO:teuthology.orchestra.run.smithi090.stdout:(12/32): boost-program-options-1.75.0-8.el9.x86 1.0 MB/s | 106 kB 00:00 2024-03-24T12:00:43.130 INFO:teuthology.orchestra.run.smithi090.stdout:(13/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-24T12:00:43.163 INFO:teuthology.orchestra.run.smithi090.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-24T12:00:43.230 INFO:teuthology.orchestra.run.smithi090.stdout:(15/32): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2024-03-24T12:00:43.273 INFO:teuthology.orchestra.run.smithi090.stdout:(16/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-03-24T12:00:43.306 INFO:teuthology.orchestra.run.smithi090.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 1.2 MB/s | 41 kB 00:00 2024-03-24T12:00:43.410 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-03-24T12:00:43.410 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:00:43.410 INFO:teuthology.orchestra.run.smithi006.stdout:Upgraded: 2024-03-24T12:00:43.410 INFO:teuthology.orchestra.run.smithi006.stdout: librados2-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:00:43.410 INFO:teuthology.orchestra.run.smithi006.stdout: librbd1-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:00:43.410 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-24T12:00:43.410 INFO:teuthology.orchestra.run.smithi006.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-24T12:00:43.410 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-base-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:00:43.411 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-common-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:00:43.411 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-radosgw-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:00:43.411 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-selinux-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:00:43.411 INFO:teuthology.orchestra.run.smithi006.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-24T12:00:43.411 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-24T12:00:43.411 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-24T12:00:43.411 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs2-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:00:43.411 INFO:teuthology.orchestra.run.smithi006.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-24T12:00:43.411 INFO:teuthology.orchestra.run.smithi006.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-24T12:00:43.411 INFO:teuthology.orchestra.run.smithi006.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-24T12:00:43.411 INFO:teuthology.orchestra.run.smithi006.stdout: libradosstriper1-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:00:43.411 INFO:teuthology.orchestra.run.smithi006.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-24T12:00:43.411 INFO:teuthology.orchestra.run.smithi006.stdout: librgw2-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:00:43.412 INFO:teuthology.orchestra.run.smithi006.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-24T12:00:43.412 INFO:teuthology.orchestra.run.smithi006.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-24T12:00:43.412 INFO:teuthology.orchestra.run.smithi006.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-24T12:00:43.412 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-argparse-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:00:43.412 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-common-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:00:43.412 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cephfs-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:00:43.412 INFO:teuthology.orchestra.run.smithi006.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-24T12:00:43.412 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rados-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:00:43.412 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rbd-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:00:43.412 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rgw-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:00:43.412 INFO:teuthology.orchestra.run.smithi006.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-24T12:00:43.412 INFO:teuthology.orchestra.run.smithi006.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-24T12:00:43.412 INFO:teuthology.orchestra.run.smithi006.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-24T12:00:43.412 INFO:teuthology.orchestra.run.smithi006.stdout: re2-1:20211101-3.el9.x86_64 2024-03-24T12:00:43.413 INFO:teuthology.orchestra.run.smithi006.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-24T12:00:43.413 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:00:43.413 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:00:43.456 INFO:teuthology.orchestra.run.smithi090.stdout:(18/32): qatlib-23.11.0-1.el9.x86_64.rpm 1.4 MB/s | 216 kB 00:00 2024-03-24T12:00:43.507 INFO:teuthology.orchestra.run.smithi090.stdout:(19/32): qatlib-service-23.11.0-1.el9.x86_64.rp 697 kB/s | 35 kB 00:00 2024-03-24T12:00:43.557 INFO:teuthology.orchestra.run.smithi090.stdout:(20/32): qatzip-libs-1.1.2-2.el9.x86_64.rpm 889 kB/s | 44 kB 00:00 2024-03-24T12:00:43.750 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-test 2024-03-24T12:00:44.075 INFO:teuthology.orchestra.run.smithi090.stdout:(21/32): ceph-radosgw-19.0.0-1667.gdb0330b1.el9 3.8 MB/s | 9.8 MB 00:02 2024-03-24T12:00:44.101 INFO:teuthology.orchestra.run.smithi090.stdout:(22/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 568 kB/s | 309 kB 00:00 2024-03-24T12:00:44.176 INFO:teuthology.orchestra.run.smithi090.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 345 kB/s | 26 kB 00:00 2024-03-24T12:00:44.252 INFO:teuthology.orchestra.run.smithi090.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 648 kB/s | 49 kB 00:00 2024-03-24T12:00:44.327 INFO:teuthology.orchestra.run.smithi090.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 897 kB/s | 67 kB 00:00 2024-03-24T12:00:44.330 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:00:30 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:00:44.486 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:00:44.486 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:00:44.486 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-03-24T12:00:44.487 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:00:44.487 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-24T12:00:44.487 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-test x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 47 M 2024-03-24T12:00:44.487 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-03-24T12:00:44.487 INFO:teuthology.orchestra.run.smithi006.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-24T12:00:44.487 INFO:teuthology.orchestra.run.smithi006.stdout: libcephsqlite x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 163 k 2024-03-24T12:00:44.487 INFO:teuthology.orchestra.run.smithi006.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-24T12:00:44.487 INFO:teuthology.orchestra.run.smithi006.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-24T12:00:44.487 INFO:teuthology.orchestra.run.smithi006.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-24T12:00:44.487 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:00:44.487 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-24T12:00:44.487 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:00:44.487 INFO:teuthology.orchestra.run.smithi006.stdout:Install 6 Packages 2024-03-24T12:00:44.487 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:00:44.488 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 48 M 2024-03-24T12:00:44.488 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 205 M 2024-03-24T12:00:44.488 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-24T12:00:44.502 INFO:teuthology.orchestra.run.smithi090.stdout:(26/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.7 MB/s | 838 kB 00:00 2024-03-24T12:00:44.578 INFO:teuthology.orchestra.run.smithi090.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-03-24T12:00:44.803 INFO:teuthology.orchestra.run.smithi090.stdout:(28/32): ceph-common-19.0.0-1667.gdb0330b1.el9. 6.0 MB/s | 20 MB 00:03 2024-03-24T12:00:44.853 INFO:teuthology.orchestra.run.smithi090.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 5.7 MB/s | 1.6 MB 00:00 2024-03-24T12:00:44.868 INFO:teuthology.orchestra.run.smithi006.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 909 kB/s | 217 kB 00:00 2024-03-24T12:00:44.937 INFO:teuthology.orchestra.run.smithi090.stdout:(30/32): libarrow-9.0.0-10.el9.x86_64.rpm 5.1 MB/s | 4.4 MB 00:00 2024-03-24T12:00:45.001 INFO:teuthology.orchestra.run.smithi006.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 2.2 MB/s | 300 kB 00:00 2024-03-24T12:00:45.068 INFO:teuthology.orchestra.run.smithi006.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 938 kB/s | 63 kB 00:00 2024-03-24T12:00:45.243 INFO:teuthology.orchestra.run.smithi006.stdout:(4/6): libcephsqlite-19.0.0-1667.gdb0330b1.el9. 265 kB/s | 163 kB 00:00 2024-03-24T12:00:45.262 INFO:teuthology.orchestra.run.smithi090.stdout:(31/32): librados2-19.0.0-1667.gdb0330b1.el9.x8 7.3 MB/s | 3.4 MB 00:00 2024-03-24T12:00:45.329 INFO:teuthology.orchestra.run.smithi090.stdout:(32/32): librbd1-19.0.0-1667.gdb0330b1.el9.x86_ 6.6 MB/s | 3.1 MB 00:00 2024-03-24T12:00:45.333 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:00:45.333 INFO:teuthology.orchestra.run.smithi090.stdout:Total 14 MB/s | 58 MB 00:04 2024-03-24T12:00:45.362 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-03-24T12:00:45.396 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-03-24T12:00:45.397 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-03-24T12:00:45.585 INFO:teuthology.orchestra.run.smithi006.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 362 kB/s | 187 kB 00:00 2024-03-24T12:00:45.910 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-03-24T12:00:45.910 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-03-24T12:00:47.324 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-03-24T12:00:47.409 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-03-24T12:00:47.451 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-03-24T12:00:47.476 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-03-24T12:00:47.507 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-03-24T12:00:47.524 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-03-24T12:00:47.741 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 5/34 2024-03-24T12:00:47.798 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 5/34 2024-03-24T12:00:48.126 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-03-24T12:00:48.213 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librados2-2:19.0.0-1667.gdb0330b1.el9.x86_64 7/34 2024-03-24T12:00:48.288 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:19.0.0-1667.gdb0330b1.el9.x86_64 7/34 2024-03-24T12:00:48.384 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rados-2:19.0.0-1667.gdb0330b1.el9.x86_64 8/34 2024-03-24T12:00:48.415 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs2-2:19.0.0-1667.gdb0330b1.el9.x86_64 9/34 2024-03-24T12:00:48.461 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephfs2-2:19.0.0-1667.gdb0330b1.el9.x86_64 9/34 2024-03-24T12:00:48.521 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-03-24T12:00:48.567 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-argparse-2:19.0.0-1667.gdb0330b1.el9. 11/34 2024-03-24T12:00:48.634 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cephfs-2:19.0.0-1667.gdb0330b1.el9.x86_64 12/34 2024-03-24T12:00:48.659 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libradosstriper1-2:19.0.0-1667.gdb0330b1.el9.x86_6 13/34 2024-03-24T12:00:48.706 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1667.gdb0330b1.el9.x86_6 13/34 2024-03-24T12:00:48.742 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 14/34 2024-03-24T12:00:48.850 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-03-24T12:00:48.894 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-03-24T12:00:48.934 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-03-24T12:00:48.959 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-03-24T12:00:48.988 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-03-24T12:00:49.021 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-03-24T12:00:49.337 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-03-24T12:00:49.367 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librbd1-2:19.0.0-1667.gdb0330b1.el9.x86_64 22/34 2024-03-24T12:00:49.449 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:19.0.0-1667.gdb0330b1.el9.x86_64 22/34 2024-03-24T12:00:49.493 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rbd-2:19.0.0-1667.gdb0330b1.el9.x86_64 23/34 2024-03-24T12:00:49.530 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-03-24T12:00:49.996 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-03-24T12:00:50.022 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librgw2-2:19.0.0-1667.gdb0330b1.el9.x86_64 26/34 2024-03-24T12:00:50.093 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librgw2-2:19.0.0-1667.gdb0330b1.el9.x86_64 26/34 2024-03-24T12:00:50.161 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rgw-2:19.0.0-1667.gdb0330b1.el9.x86_64 27/34 2024-03-24T12:00:50.191 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-common-2:19.0.0-1667.gdb0330b1.el9.x8 28/34 2024-03-24T12:00:52.386 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:19.0.0-1667.gdb0330b1.el9.x86_64 29/34 2024-03-24T12:00:52.431 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-common-2:19.0.0-1667.gdb0330b1.el9.x86_64 29/34 2024-03-24T12:00:53.031 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:19.0.0-1667.gdb0330b1.el9.x86_64 29/34 2024-03-24T12:00:53.077 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-base-2:19.0.0-1667.gdb0330b1.el9.x86_64 30/34 2024-03-24T12:00:53.207 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-base-2:19.0.0-1667.gdb0330b1.el9.x86_64 30/34 2024-03-24T12:00:53.207 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-24T12:00:53.207 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-24T12:00:53.208 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:00:53.245 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-selinux-2:19.0.0-1667.gdb0330b1.el9.x86_64 31/34 2024-03-24T12:00:54.010 INFO:teuthology.orchestra.run.smithi006.stdout:(6/6): ceph-test-19.0.0-1667.gdb0330b1.el9.x86_ 5.0 MB/s | 47 MB 00:09 2024-03-24T12:00:54.013 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:00:54.014 INFO:teuthology.orchestra.run.smithi006.stdout:Total 5.1 MB/s | 48 MB 00:09 2024-03-24T12:00:54.029 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-24T12:00:54.053 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-24T12:00:54.053 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-24T12:00:54.455 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-24T12:00:54.456 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-24T12:00:55.048 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-24T12:00:55.121 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-24T12:00:55.172 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-24T12:00:55.212 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-24T12:00:55.258 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-24T12:00:55.289 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephsqlite-2:19.0.0-1667.gdb0330b1.el9.x86_64 5/6 2024-03-24T12:01:00.037 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1667.gdb0330b1.el9.x86_64 5/6 2024-03-24T12:01:00.124 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-test-2:19.0.0-1667.gdb0330b1.el9.x86_64 6/6 2024-03-24T12:01:02.489 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-test-2:19.0.0-1667.gdb0330b1.el9.x86_64 6/6 2024-03-24T12:01:02.490 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-test-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/6 2024-03-24T12:01:02.490 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephsqlite-2:19.0.0-1667.gdb0330b1.el9.x86_64 2/6 2024-03-24T12:01:02.490 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-24T12:01:02.490 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-24T12:01:02.490 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-24T12:01:02.894 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-24T12:01:02.894 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:02.894 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-24T12:01:02.894 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-test-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:02.894 INFO:teuthology.orchestra.run.smithi006.stdout: jq-1.6-16.el9.x86_64 2024-03-24T12:01:02.894 INFO:teuthology.orchestra.run.smithi006.stdout: libcephsqlite-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:02.894 INFO:teuthology.orchestra.run.smithi006.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-24T12:01:02.894 INFO:teuthology.orchestra.run.smithi006.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-24T12:01:02.894 INFO:teuthology.orchestra.run.smithi006.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-24T12:01:02.894 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:02.895 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:01:03.113 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph 2024-03-24T12:01:03.679 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:00:49 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:01:03.754 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1667.gdb0330b1.el9.x86_64 31/34 2024-03-24T12:01:03.754 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /sys 2024-03-24T12:01:03.754 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /proc 2024-03-24T12:01:03.755 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /mnt 2024-03-24T12:01:03.755 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /var/tmp 2024-03-24T12:01:03.755 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /home 2024-03-24T12:01:03.755 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /root 2024-03-24T12:01:03.755 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /tmp 2024-03-24T12:01:03.755 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:03.846 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:01:03.849 INFO:teuthology.orchestra.run.smithi006.stdout:========================================================================================= 2024-03-24T12:01:03.849 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-03-24T12:01:03.849 INFO:teuthology.orchestra.run.smithi006.stdout:========================================================================================= 2024-03-24T12:01:03.849 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-24T12:01:03.849 INFO:teuthology.orchestra.run.smithi006.stdout: ceph x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 6.6 k 2024-03-24T12:01:03.849 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-03-24T12:01:03.849 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mds x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 2.1 M 2024-03-24T12:01:03.849 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 1.5 M 2024-03-24T12:01:03.849 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1667.gdb0330b1.el9 ceph-noarch 247 k 2024-03-24T12:01:03.849 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mon x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 4.7 M 2024-03-24T12:01:03.850 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-osd x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 17 M 2024-03-24T12:01:03.850 INFO:teuthology.orchestra.run.smithi006.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-24T12:01:03.850 INFO:teuthology.orchestra.run.smithi006.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-24T12:01:03.850 INFO:teuthology.orchestra.run.smithi006.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-24T12:01:03.850 INFO:teuthology.orchestra.run.smithi006.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-24T12:01:03.850 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-24T12:01:03.850 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-24T12:01:03.850 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-24T12:01:03.850 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-24T12:01:03.850 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-24T12:01:03.850 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-24T12:01:03.850 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-24T12:01:03.850 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-24T12:01:03.851 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-24T12:01:03.851 INFO:teuthology.orchestra.run.smithi006.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-24T12:01:03.851 INFO:teuthology.orchestra.run.smithi006.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-24T12:01:03.851 INFO:teuthology.orchestra.run.smithi006.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-24T12:01:03.851 INFO:teuthology.orchestra.run.smithi006.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-24T12:01:03.851 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-24T12:01:03.851 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-24T12:01:03.851 INFO:teuthology.orchestra.run.smithi006.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-24T12:01:03.851 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-24T12:01:03.852 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-24T12:01:03.852 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-24T12:01:03.852 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-24T12:01:03.852 INFO:teuthology.orchestra.run.smithi006.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-24T12:01:03.852 INFO:teuthology.orchestra.run.smithi006.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-24T12:01:03.852 INFO:teuthology.orchestra.run.smithi006.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-24T12:01:03.852 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-24T12:01:03.852 INFO:teuthology.orchestra.run.smithi006.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-24T12:01:03.852 INFO:teuthology.orchestra.run.smithi006.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-24T12:01:03.852 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:03.852 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-24T12:01:03.852 INFO:teuthology.orchestra.run.smithi006.stdout:========================================================================================= 2024-03-24T12:01:03.852 INFO:teuthology.orchestra.run.smithi006.stdout:Install 36 Packages 2024-03-24T12:01:03.852 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:03.853 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 30 M 2024-03-24T12:01:03.853 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 107 M 2024-03-24T12:01:03.853 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-24T12:01:04.382 INFO:teuthology.orchestra.run.smithi006.stdout:(1/36): ceph-19.0.0-1667.gdb0330b1.el9.x86_64.r 28 kB/s | 6.6 kB 00:00 2024-03-24T12:01:04.532 INFO:teuthology.orchestra.run.smithi006.stdout:(2/36): ceph-mds-19.0.0-1667.gdb0330b1.el9.x86_ 5.5 MB/s | 2.1 MB 00:00 2024-03-24T12:01:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-radosgw-2:19.0.0-1667.gdb0330b1.el9.x86_64 32/34 2024-03-24T12:01:04.671 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1667.gdb0330b1.el9.x86_64 32/34 2024-03-24T12:01:04.671 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T12:01:04.671 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-24T12:01:04.672 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-24T12:01:04.672 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-24T12:01:04.672 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:04.673 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-24T12:01:04.758 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-24T12:01:04.758 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-24T12:01:04.891 INFO:teuthology.orchestra.run.smithi006.stdout:(3/36): ceph-mgr-19.0.0-1667.gdb0330b1.el9.x86_ 2.0 MB/s | 1.5 MB 00:00 2024-03-24T12:01:05.007 INFO:teuthology.orchestra.run.smithi006.stdout:(4/36): ceph-mgr-modules-core-19.0.0-1667.gdb03 2.1 MB/s | 247 kB 00:00 2024-03-24T12:01:05.124 INFO:teuthology.orchestra.run.smithi006.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 187 kB 00:00 2024-03-24T12:01:05.158 INFO:teuthology.orchestra.run.smithi006.stdout:(6/36): python3-mako-1.1.4-6.el9.noarch.rpm 4.2 MB/s | 144 kB 00:00 2024-03-24T12:01:05.192 INFO:teuthology.orchestra.run.smithi006.stdout:(7/36): python3-markupsafe-1.1.1-12.el9.x86_64. 948 kB/s | 32 kB 00:00 2024-03-24T12:01:05.375 INFO:teuthology.orchestra.run.smithi006.stdout:(8/36): ceph-osd-19.0.0-1667.gdb0330b1.el9.x86_ 20 MB/s | 17 MB 00:00 2024-03-24T12:01:05.400 INFO:teuthology.orchestra.run.smithi006.stdout:(9/36): python3-pytz-2021.1-5.el9.noarch.rpm 226 kB/s | 47 kB 00:00 2024-03-24T12:01:05.426 INFO:teuthology.orchestra.run.smithi006.stdout:(10/36): python3-toml-0.10.2-6.el9.noarch.rpm 773 kB/s | 39 kB 00:00 2024-03-24T12:01:05.451 INFO:teuthology.orchestra.run.smithi006.stdout:(11/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-24T12:01:05.518 INFO:teuthology.orchestra.run.smithi006.stdout:(12/36): python3-cryptography-36.0.1-4.el9.x86_ 13 MB/s | 1.1 MB 00:00 2024-03-24T12:01:05.543 INFO:teuthology.orchestra.run.smithi006.stdout:(13/36): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-03-24T12:01:05.568 INFO:teuthology.orchestra.run.smithi006.stdout:(14/36): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2024-03-24T12:01:05.585 INFO:teuthology.orchestra.run.smithi006.stdout:(15/36): python3-urllib3-1.26.5-5.el9.noarch.rp 4.4 MB/s | 187 kB 00:00 2024-03-24T12:01:05.644 INFO:teuthology.orchestra.run.smithi006.stdout:(16/36): python3-requests-2.25.1-8.el9.noarch.r 1.6 MB/s | 125 kB 00:00 2024-03-24T12:01:05.869 INFO:teuthology.orchestra.run.smithi006.stdout:(17/36): luarocks-3.9.2-1.el9.noarch.rpm 672 kB/s | 151 kB 00:00 2024-03-24T12:01:05.919 INFO:teuthology.orchestra.run.smithi006.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 871 kB/s | 43 kB 00:00 2024-03-24T12:01:06.034 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-24T12:01:06.034 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-base-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/34 2024-03-24T12:01:06.034 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-common-2:19.0.0-1667.gdb0330b1.el9.x86_64 2/34 2024-03-24T12:01:06.034 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-radosgw-2:19.0.0-1667.gdb0330b1.el9.x86_64 3/34 2024-03-24T12:01:06.034 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-selinux-2:19.0.0-1667.gdb0330b1.el9.x86_64 4/34 2024-03-24T12:01:06.035 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs2-2:19.0.0-1667.gdb0330b1.el9.x86_64 5/34 2024-03-24T12:01:06.035 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libradosstriper1-2:19.0.0-1667.gdb0330b1.el9.x86_6 6/34 2024-03-24T12:01:06.035 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librgw2-2:19.0.0-1667.gdb0330b1.el9.x86_64 7/34 2024-03-24T12:01:06.035 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1667.gdb0330b1.el9. 8/34 2024-03-24T12:01:06.035 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-common-2:19.0.0-1667.gdb0330b1.el9.x8 9/34 2024-03-24T12:01:06.035 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cephfs-2:19.0.0-1667.gdb0330b1.el9.x86_64 10/34 2024-03-24T12:01:06.035 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rados-2:19.0.0-1667.gdb0330b1.el9.x86_64 11/34 2024-03-24T12:01:06.035 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rbd-2:19.0.0-1667.gdb0330b1.el9.x86_64 12/34 2024-03-24T12:01:06.035 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rgw-2:19.0.0-1667.gdb0330b1.el9.x86_64 13/34 2024-03-24T12:01:06.035 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-03-24T12:01:06.035 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-03-24T12:01:06.035 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-03-24T12:01:06.035 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-03-24T12:01:06.035 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-03-24T12:01:06.035 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-03-24T12:01:06.036 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 20/34 2024-03-24T12:01:06.036 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 21/34 2024-03-24T12:01:06.036 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 22/34 2024-03-24T12:01:06.036 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-03-24T12:01:06.036 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-03-24T12:01:06.036 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-03-24T12:01:06.036 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-03-24T12:01:06.036 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-03-24T12:01:06.036 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-03-24T12:01:06.036 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-03-24T12:01:06.036 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-03-24T12:01:06.037 INFO:teuthology.orchestra.run.smithi006.stdout:(19/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-03-24T12:01:06.037 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:19.0.0-1667.gdb0330b1.el9.x86_64 31/34 2024-03-24T12:01:06.037 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-03-24T12:01:06.037 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:19.0.0-1667.gdb0330b1.el9.x86_64 33/34 2024-03-24T12:01:06.162 INFO:teuthology.orchestra.run.smithi006.stdout:(20/36): lua-devel-5.4.4-4.el9.x86_64.rpm 39 kB/s | 22 kB 00:00 2024-03-24T12:01:06.221 INFO:teuthology.orchestra.run.smithi006.stdout:(21/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.9 MB/s | 358 kB 00:00 2024-03-24T12:01:06.254 INFO:teuthology.orchestra.run.smithi006.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 117 kB/s | 11 kB 00:00 2024-03-24T12:01:06.271 INFO:teuthology.orchestra.run.smithi006.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 351 kB/s | 18 kB 00:00 2024-03-24T12:01:06.305 INFO:teuthology.orchestra.run.smithi006.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-03-24T12:01:06.322 INFO:teuthology.orchestra.run.smithi006.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-03-24T12:01:06.389 INFO:teuthology.orchestra.run.smithi006.stdout:(26/36): ceph-mon-19.0.0-1667.gdb0330b1.el9.x86 2.3 MB/s | 4.7 MB 00:02 2024-03-24T12:01:06.406 INFO:teuthology.orchestra.run.smithi006.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 196 kB/s | 20 kB 00:00 2024-03-24T12:01:06.422 INFO:teuthology.orchestra.run.smithi006.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 460 kB/s | 46 kB 00:00 2024-03-24T12:01:06.473 INFO:teuthology.orchestra.run.smithi006.stdout:(29/36): python3-portend-3.1.0-2.el9.noarch.rpm 329 kB/s | 16 kB 00:00 2024-03-24T12:01:06.532 INFO:teuthology.orchestra.run.smithi006.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-03-24T12:01:06.565 INFO:teuthology.orchestra.run.smithi006.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 979 kB/s | 90 kB 00:00 2024-03-24T12:01:06.590 INFO:teuthology.orchestra.run.smithi006.stdout:(32/36): python3-more-itertools-8.12.0-2.el9.no 392 kB/s | 79 kB 00:00 2024-03-24T12:01:06.607 INFO:teuthology.orchestra.run.smithi006.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 475 kB/s | 36 kB 00:00 2024-03-24T12:01:06.657 INFO:teuthology.orchestra.run.smithi006.stdout:(34/36): python3-zc-lockfile-2.0-10.el9.noarch. 401 kB/s | 20 kB 00:00 2024-03-24T12:01:06.716 INFO:teuthology.orchestra.run.smithi006.stdout:(35/36): python3-webob-1.8.7-6.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2024-03-24T12:01:06.841 INFO:teuthology.orchestra.run.smithi006.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.7 MB/s | 427 kB 00:00 2024-03-24T12:01:06.845 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:01:06.845 INFO:teuthology.orchestra.run.smithi006.stdout:Total 9.9 MB/s | 30 MB 00:02 2024-03-24T12:01:06.870 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-24T12:01:06.916 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-24T12:01:06.917 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-24T12:01:07.152 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-03-24T12:01:07.153 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:07.153 INFO:teuthology.orchestra.run.smithi090.stdout:Upgraded: 2024-03-24T12:01:07.153 INFO:teuthology.orchestra.run.smithi090.stdout: librados2-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:07.153 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:07.153 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-03-24T12:01:07.153 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-24T12:01:07.153 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:07.153 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:07.153 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:07.153 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:07.153 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-24T12:01:07.153 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-24T12:01:07.153 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-24T12:01:07.153 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:07.154 INFO:teuthology.orchestra.run.smithi090.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-24T12:01:07.154 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-24T12:01:07.154 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-24T12:01:07.154 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:07.154 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-24T12:01:07.154 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:07.154 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-24T12:01:07.154 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-24T12:01:07.154 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-24T12:01:07.154 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:07.154 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:07.154 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:07.154 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-24T12:01:07.155 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:07.155 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:07.155 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:07.155 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-24T12:01:07.155 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-24T12:01:07.155 INFO:teuthology.orchestra.run.smithi090.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-24T12:01:07.155 INFO:teuthology.orchestra.run.smithi090.stdout: re2-1:20211101-3.el9.x86_64 2024-03-24T12:01:07.155 INFO:teuthology.orchestra.run.smithi090.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-24T12:01:07.155 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:07.155 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:01:07.343 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-24T12:01:07.344 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-24T12:01:07.576 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-test 2024-03-24T12:01:08.136 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:30 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:01:08.314 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:01:08.314 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:01:08.314 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-03-24T12:01:08.314 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:01:08.314 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-03-24T12:01:08.314 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 47 M 2024-03-24T12:01:08.314 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-03-24T12:01:08.314 INFO:teuthology.orchestra.run.smithi090.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-24T12:01:08.315 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 163 k 2024-03-24T12:01:08.315 INFO:teuthology.orchestra.run.smithi090.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-24T12:01:08.315 INFO:teuthology.orchestra.run.smithi090.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-24T12:01:08.315 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-24T12:01:08.315 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:08.315 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-03-24T12:01:08.315 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:01:08.315 INFO:teuthology.orchestra.run.smithi090.stdout:Install 6 Packages 2024-03-24T12:01:08.315 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:08.316 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 48 M 2024-03-24T12:01:08.316 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 205 M 2024-03-24T12:01:08.316 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-03-24T12:01:08.654 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-24T12:01:08.727 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-03-24T12:01:08.760 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-03-24T12:01:08.762 INFO:teuthology.orchestra.run.smithi090.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.3 MB/s | 217 kB 00:00 2024-03-24T12:01:08.788 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-03-24T12:01:08.845 INFO:teuthology.orchestra.run.smithi090.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 3.5 MB/s | 300 kB 00:00 2024-03-24T12:01:08.860 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-03-24T12:01:08.871 INFO:teuthology.orchestra.run.smithi090.stdout:(3/6): libcephsqlite-19.0.0-1667.gdb0330b1.el9. 592 kB/s | 163 kB 00:00 2024-03-24T12:01:08.896 INFO:teuthology.orchestra.run.smithi090.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-24T12:01:08.903 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-03-24T12:01:08.936 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-03-24T12:01:09.040 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-03-24T12:01:09.118 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-03-24T12:01:09.164 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-03-24T12:01:09.247 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-03-24T12:01:09.288 INFO:teuthology.orchestra.run.smithi090.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 449 kB/s | 187 kB 00:00 2024-03-24T12:01:09.299 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-03-24T12:01:09.337 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-03-24T12:01:09.386 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-03-24T12:01:09.449 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-03-24T12:01:09.643 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-03-24T12:01:09.691 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-03-24T12:01:09.731 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-03-24T12:01:09.764 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-03-24T12:01:09.786 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-03-24T12:01:09.857 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-03-24T12:01:09.888 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-24T12:01:09.921 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-03-24T12:01:09.962 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-03-24T12:01:09.996 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-03-24T12:01:10.039 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-03-24T12:01:10.132 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-03-24T12:01:10.178 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-03-24T12:01:10.253 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-03-24T12:01:10.394 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-03-24T12:01:10.525 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-24T12:01:10.694 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1667.gdb0330b1.el9. 31/36 2024-03-24T12:01:10.716 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-2:19.0.0-1667.gdb0330b1.el9.x86_64 32/36 2024-03-24T12:01:10.742 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1667.gdb0330b1.el9.x86_64 32/36 2024-03-24T12:01:10.742 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T12:01:10.742 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-24T12:01:10.742 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-24T12:01:10.742 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-24T12:01:10.742 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:11.297 INFO:teuthology.orchestra.run.smithi090.stdout:(6/6): ceph-test-19.0.0-1667.gdb0330b1.el9.x86_ 18 MB/s | 47 MB 00:02 2024-03-24T12:01:11.300 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:01:11.300 INFO:teuthology.orchestra.run.smithi090.stdout:Total 16 MB/s | 48 MB 00:02 2024-03-24T12:01:11.314 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-03-24T12:01:11.334 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-03-24T12:01:11.334 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-03-24T12:01:11.719 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-03-24T12:01:11.719 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-03-24T12:01:12.267 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-osd-2:19.0.0-1667.gdb0330b1.el9.x86_64 33/36 2024-03-24T12:01:12.293 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-osd-2:19.0.0-1667.gdb0330b1.el9.x86_64 33/36 2024-03-24T12:01:12.293 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T12:01:12.293 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-24T12:01:12.293 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-24T12:01:12.293 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-24T12:01:12.294 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:12.312 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-03-24T12:01:12.399 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-24T12:01:12.442 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-24T12:01:12.485 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-24T12:01:12.533 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-24T12:01:12.560 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephsqlite-2:19.0.0-1667.gdb0330b1.el9.x86_64 5/6 2024-03-24T12:01:12.759 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mon-2:19.0.0-1667.gdb0330b1.el9.x86_64 34/36 2024-03-24T12:01:12.783 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mon-2:19.0.0-1667.gdb0330b1.el9.x86_64 34/36 2024-03-24T12:01:12.783 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T12:01:12.783 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-24T12:01:12.783 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-24T12:01:12.783 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-24T12:01:12.783 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:13.001 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mds-2:19.0.0-1667.gdb0330b1.el9.x86_64 35/36 2024-03-24T12:01:13.024 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mds-2:19.0.0-1667.gdb0330b1.el9.x86_64 35/36 2024-03-24T12:01:13.024 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T12:01:13.024 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-24T12:01:13.025 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-24T12:01:13.025 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-24T12:01:13.025 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:13.094 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-2:19.0.0-1667.gdb0330b1.el9.x86_64 36/36 2024-03-24T12:01:14.773 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-2:19.0.0-1667.gdb0330b1.el9.x86_64 36/36 2024-03-24T12:01:14.774 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/36 2024-03-24T12:01:14.774 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mds-2:19.0.0-1667.gdb0330b1.el9.x86_64 2/36 2024-03-24T12:01:14.774 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-2:19.0.0-1667.gdb0330b1.el9.x86_64 3/36 2024-03-24T12:01:14.774 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mon-2:19.0.0-1667.gdb0330b1.el9.x86_64 4/36 2024-03-24T12:01:14.774 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-osd-2:19.0.0-1667.gdb0330b1.el9.x86_64 5/36 2024-03-24T12:01:14.774 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1667.gdb0330b1.el9. 6/36 2024-03-24T12:01:14.774 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-03-24T12:01:14.774 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-03-24T12:01:14.774 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-03-24T12:01:14.774 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-03-24T12:01:14.774 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-03-24T12:01:14.774 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-03-24T12:01:14.774 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-03-24T12:01:14.774 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-03-24T12:01:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-03-24T12:01:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/36 2024-03-24T12:01:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 17/36 2024-03-24T12:01:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-03-24T12:01:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-03-24T12:01:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-03-24T12:01:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-24T12:01:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-03-24T12:01:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-03-24T12:01:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-03-24T12:01:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-03-24T12:01:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-03-24T12:01:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-03-24T12:01:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-03-24T12:01:14.775 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-03-24T12:01:14.776 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-24T12:01:14.776 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-03-24T12:01:14.776 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-03-24T12:01:14.776 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-03-24T12:01:14.776 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-03-24T12:01:14.776 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-03-24T12:01:15.771 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-03-24T12:01:15.771 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:15.771 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-24T12:01:15.771 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:15.771 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mds-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:15.771 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:15.771 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-modules-core-2:19.0.0-1667.gdb0330b1.el9.noarch 2024-03-24T12:01:15.771 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mon-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:15.771 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-osd-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:15.771 INFO:teuthology.orchestra.run.smithi006.stdout: lua-5.4.4-4.el9.x86_64 2024-03-24T12:01:15.771 INFO:teuthology.orchestra.run.smithi006.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-24T12:01:15.771 INFO:teuthology.orchestra.run.smithi006.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-24T12:01:15.771 INFO:teuthology.orchestra.run.smithi006.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-24T12:01:15.772 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-24T12:01:15.772 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-24T12:01:15.772 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-24T12:01:15.772 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-24T12:01:15.772 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-24T12:01:15.772 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-24T12:01:15.772 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-24T12:01:15.772 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-24T12:01:15.772 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-24T12:01:15.772 INFO:teuthology.orchestra.run.smithi006.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-24T12:01:15.772 INFO:teuthology.orchestra.run.smithi006.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-24T12:01:15.772 INFO:teuthology.orchestra.run.smithi006.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-24T12:01:15.772 INFO:teuthology.orchestra.run.smithi006.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-24T12:01:15.772 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-24T12:01:15.773 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ply-3.11-14.el9.noarch 2024-03-24T12:01:15.773 INFO:teuthology.orchestra.run.smithi006.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-24T12:01:15.773 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-24T12:01:15.773 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-24T12:01:15.773 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-24T12:01:15.773 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-24T12:01:15.773 INFO:teuthology.orchestra.run.smithi006.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-24T12:01:15.773 INFO:teuthology.orchestra.run.smithi006.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-24T12:01:15.773 INFO:teuthology.orchestra.run.smithi006.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-24T12:01:15.773 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-24T12:01:15.773 INFO:teuthology.orchestra.run.smithi006.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-24T12:01:15.773 INFO:teuthology.orchestra.run.smithi006.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-24T12:01:15.773 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:15.773 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:01:16.139 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-base 2024-03-24T12:01:16.717 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:02 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:01:16.819 INFO:teuthology.orchestra.run.smithi006.stdout:Package ceph-base-2:19.0.0-1667.gdb0330b1.el9.x86_64 is already installed. 2024-03-24T12:01:16.873 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:01:16.874 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2024-03-24T12:01:16.874 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:01:16.938 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install cephadm 2024-03-24T12:01:17.036 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1667.gdb0330b1.el9.x86_64 5/6 2024-03-24T12:01:17.112 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-test-2:19.0.0-1667.gdb0330b1.el9.x86_64 6/6 2024-03-24T12:01:17.508 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:03 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:01:17.661 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:01:17.661 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:01:17.661 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-03-24T12:01:17.662 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:01:17.662 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-24T12:01:17.662 INFO:teuthology.orchestra.run.smithi006.stdout: cephadm noarch 2:19.0.0-1667.gdb0330b1.el9 ceph-noarch 748 k 2024-03-24T12:01:17.662 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:17.662 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-24T12:01:17.662 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:01:17.662 INFO:teuthology.orchestra.run.smithi006.stdout:Install 1 Package 2024-03-24T12:01:17.662 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:17.662 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 748 k 2024-03-24T12:01:17.662 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 754 k 2024-03-24T12:01:17.662 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-24T12:01:17.934 INFO:teuthology.orchestra.run.smithi006.stdout:cephadm-19.0.0-1667.gdb0330b1.el9.noarch.rpm 2.7 MB/s | 748 kB 00:00 2024-03-24T12:01:17.935 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:01:17.935 INFO:teuthology.orchestra.run.smithi006.stdout:Total 2.7 MB/s | 748 kB 00:00 2024-03-24T12:01:17.935 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-24T12:01:17.940 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-24T12:01:17.940 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-24T12:01:17.946 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-24T12:01:17.947 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-24T12:01:18.037 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-24T12:01:18.399 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: cephadm-2:19.0.0-1667.gdb0330b1.el9.noarch 1/1 2024-03-24T12:01:18.490 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephadm-2:19.0.0-1667.gdb0330b1.el9.noarch 1/1 2024-03-24T12:01:18.874 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: cephadm-2:19.0.0-1667.gdb0330b1.el9.noarch 1/1 2024-03-24T12:01:19.145 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephadm-2:19.0.0-1667.gdb0330b1.el9.noarch 1/1 2024-03-24T12:01:19.146 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:19.146 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-24T12:01:19.146 INFO:teuthology.orchestra.run.smithi006.stdout: cephadm-2:19.0.0-1667.gdb0330b1.el9.noarch 2024-03-24T12:01:19.146 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:19.146 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:01:19.363 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-immutable-object-cache 2024-03-24T12:01:19.932 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:05 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:01:20.086 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:01:20.086 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:01:20.087 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repo Size 2024-03-24T12:01:20.087 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:01:20.087 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-24T12:01:20.087 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 145 k 2024-03-24T12:01:20.087 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:20.087 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-24T12:01:20.087 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:01:20.087 INFO:teuthology.orchestra.run.smithi006.stdout:Install 1 Package 2024-03-24T12:01:20.087 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:20.087 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 145 k 2024-03-24T12:01:20.087 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 436 k 2024-03-24T12:01:20.087 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-24T12:01:20.354 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-test-2:19.0.0-1667.gdb0330b1.el9.x86_64 6/6 2024-03-24T12:01:20.354 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-test-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/6 2024-03-24T12:01:20.354 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephsqlite-2:19.0.0-1667.gdb0330b1.el9.x86_64 2/6 2024-03-24T12:01:20.355 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-24T12:01:20.355 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-24T12:01:20.355 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-24T12:01:20.465 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-immutable-object-cache-19.0.0-1667.gdb0330 385 kB/s | 145 kB 00:00 2024-03-24T12:01:20.465 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:01:20.465 INFO:teuthology.orchestra.run.smithi006.stdout:Total 384 kB/s | 145 kB 00:00 2024-03-24T12:01:20.465 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-24T12:01:20.474 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-24T12:01:20.474 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-24T12:01:20.528 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-24T12:01:20.528 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-24T12:01:20.678 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-24T12:01:20.732 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1667.gdb0330b1. 1/1 2024-03-24T12:01:20.756 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1667.gdb0330b1. 1/1 2024-03-24T12:01:20.756 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T12:01:20.756 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-24T12:01:20.756 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:20.801 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-24T12:01:20.801 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:20.801 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-03-24T12:01:20.801 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:20.801 INFO:teuthology.orchestra.run.smithi090.stdout: jq-1.6-16.el9.x86_64 2024-03-24T12:01:20.801 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:20.801 INFO:teuthology.orchestra.run.smithi090.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-24T12:01:20.802 INFO:teuthology.orchestra.run.smithi090.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-24T12:01:20.802 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-24T12:01:20.802 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:20.802 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:01:21.089 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph 2024-03-24T12:01:21.646 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:43 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:01:21.688 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1667.gdb0330b1. 1/1 2024-03-24T12:01:21.688 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:21.688 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-24T12:01:21.688 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-immutable-object-cache-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:21.688 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:21.688 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:01:21.800 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:01:21.802 INFO:teuthology.orchestra.run.smithi090.stdout:========================================================================================= 2024-03-24T12:01:21.803 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-03-24T12:01:21.803 INFO:teuthology.orchestra.run.smithi090.stdout:========================================================================================= 2024-03-24T12:01:21.803 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-03-24T12:01:21.803 INFO:teuthology.orchestra.run.smithi090.stdout: ceph x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 6.6 k 2024-03-24T12:01:21.803 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-03-24T12:01:21.803 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 2.1 M 2024-03-24T12:01:21.803 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 1.5 M 2024-03-24T12:01:21.803 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1667.gdb0330b1.el9 ceph-noarch 247 k 2024-03-24T12:01:21.803 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 4.7 M 2024-03-24T12:01:21.803 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 17 M 2024-03-24T12:01:21.803 INFO:teuthology.orchestra.run.smithi090.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-24T12:01:21.803 INFO:teuthology.orchestra.run.smithi090.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-24T12:01:21.803 INFO:teuthology.orchestra.run.smithi090.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-24T12:01:21.803 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-24T12:01:21.804 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-24T12:01:21.804 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-24T12:01:21.804 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-24T12:01:21.804 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-24T12:01:21.804 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-24T12:01:21.804 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-24T12:01:21.804 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-24T12:01:21.804 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-24T12:01:21.804 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-24T12:01:21.804 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-24T12:01:21.804 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-24T12:01:21.804 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-24T12:01:21.804 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-24T12:01:21.804 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-24T12:01:21.805 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-24T12:01:21.805 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-24T12:01:21.805 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-24T12:01:21.805 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-24T12:01:21.805 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-24T12:01:21.805 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-24T12:01:21.805 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-24T12:01:21.805 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-24T12:01:21.805 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-24T12:01:21.805 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-24T12:01:21.805 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-24T12:01:21.805 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-24T12:01:21.805 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:21.805 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-03-24T12:01:21.805 INFO:teuthology.orchestra.run.smithi090.stdout:========================================================================================= 2024-03-24T12:01:21.806 INFO:teuthology.orchestra.run.smithi090.stdout:Install 36 Packages 2024-03-24T12:01:21.806 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:21.806 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 30 M 2024-03-24T12:01:21.806 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 107 M 2024-03-24T12:01:21.806 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-03-24T12:01:21.957 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-mgr 2024-03-24T12:01:22.194 INFO:teuthology.orchestra.run.smithi090.stdout:(1/36): ceph-19.0.0-1667.gdb0330b1.el9.x86_64.r 29 kB/s | 6.6 kB 00:00 2024-03-24T12:01:22.327 INFO:teuthology.orchestra.run.smithi090.stdout:(2/36): ceph-mgr-19.0.0-1667.gdb0330b1.el9.x86_ 4.1 MB/s | 1.5 MB 00:00 2024-03-24T12:01:22.533 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:08 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:01:22.594 INFO:teuthology.orchestra.run.smithi090.stdout:(3/36): ceph-mds-19.0.0-1667.gdb0330b1.el9.x86_ 3.4 MB/s | 2.1 MB 00:00 2024-03-24T12:01:22.632 INFO:teuthology.orchestra.run.smithi006.stdout:Package ceph-mgr-2:19.0.0-1667.gdb0330b1.el9.x86_64 is already installed. 2024-03-24T12:01:22.678 INFO:teuthology.orchestra.run.smithi090.stdout:(4/36): ceph-mgr-modules-core-19.0.0-1667.gdb03 2.9 MB/s | 247 kB 00:00 2024-03-24T12:01:22.686 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:01:22.687 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2024-03-24T12:01:22.687 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:01:22.760 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-mgr-dashboard 2024-03-24T12:01:22.787 INFO:teuthology.orchestra.run.smithi090.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 187 kB 00:00 2024-03-24T12:01:22.829 INFO:teuthology.orchestra.run.smithi090.stdout:(6/36): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-24T12:01:22.862 INFO:teuthology.orchestra.run.smithi090.stdout:(7/36): python3-markupsafe-1.1.1-12.el9.x86_64. 940 kB/s | 32 kB 00:00 2024-03-24T12:01:22.896 INFO:teuthology.orchestra.run.smithi090.stdout:(8/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2024-03-24T12:01:22.930 INFO:teuthology.orchestra.run.smithi090.stdout:(9/36): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-03-24T12:01:22.972 INFO:teuthology.orchestra.run.smithi090.stdout:(10/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.6 MB/s | 241 kB 00:00 2024-03-24T12:01:23.039 INFO:teuthology.orchestra.run.smithi090.stdout:(11/36): python3-cryptography-36.0.1-4.el9.x86_ 17 MB/s | 1.1 MB 00:00 2024-03-24T12:01:23.073 INFO:teuthology.orchestra.run.smithi090.stdout:(12/36): python3-ply-3.11-14.el9.noarch.rpm 3.0 MB/s | 103 kB 00:00 2024-03-24T12:01:23.106 INFO:teuthology.orchestra.run.smithi090.stdout:(13/36): python3-pycparser-2.20-6.el9.noarch.rp 3.6 MB/s | 124 kB 00:00 2024-03-24T12:01:23.182 INFO:teuthology.orchestra.run.smithi090.stdout:(14/36): ceph-mon-19.0.0-1667.gdb0330b1.el9.x86 4.7 MB/s | 4.7 MB 00:00 2024-03-24T12:01:23.207 INFO:teuthology.orchestra.run.smithi090.stdout:(15/36): python3-urllib3-1.26.5-5.el9.noarch.rp 1.8 MB/s | 187 kB 00:00 2024-03-24T12:01:23.328 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:09 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:01:23.399 INFO:teuthology.orchestra.run.smithi090.stdout:(16/36): ceph-osd-19.0.0-1667.gdb0330b1.el9.x86 16 MB/s | 17 MB 00:01 2024-03-24T12:01:23.449 INFO:teuthology.orchestra.run.smithi090.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 92 kB/s | 22 kB 00:00 2024-03-24T12:01:23.484 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:01:23.486 INFO:teuthology.orchestra.run.smithi006.stdout:====================================================================================== 2024-03-24T12:01:23.486 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-03-24T12:01:23.486 INFO:teuthology.orchestra.run.smithi006.stdout:====================================================================================== 2024-03-24T12:01:23.486 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-24T12:01:23.486 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1667.gdb0330b1.el9 ceph-noarch 1.8 M 2024-03-24T12:01:23.486 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-03-24T12:01:23.486 INFO:teuthology.orchestra.run.smithi006.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-24T12:01:23.486 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1667.gdb0330b1.el9 ceph-noarch 31 k 2024-03-24T12:01:23.486 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1667.gdb0330b1.el9 ceph-noarch 15 k 2024-03-24T12:01:23.486 INFO:teuthology.orchestra.run.smithi006.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-24T12:01:23.487 INFO:teuthology.orchestra.run.smithi006.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-24T12:01:23.487 INFO:teuthology.orchestra.run.smithi006.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-24T12:01:23.487 INFO:teuthology.orchestra.run.smithi006.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-24T12:01:23.487 INFO:teuthology.orchestra.run.smithi006.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-24T12:01:23.487 INFO:teuthology.orchestra.run.smithi006.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-24T12:01:23.487 INFO:teuthology.orchestra.run.smithi006.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-24T12:01:23.487 INFO:teuthology.orchestra.run.smithi006.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-24T12:01:23.487 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:23.487 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-24T12:01:23.487 INFO:teuthology.orchestra.run.smithi006.stdout:====================================================================================== 2024-03-24T12:01:23.487 INFO:teuthology.orchestra.run.smithi006.stdout:Install 12 Packages 2024-03-24T12:01:23.487 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:23.488 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 6.9 M 2024-03-24T12:01:23.488 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 26 M 2024-03-24T12:01:23.488 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-24T12:01:23.508 INFO:teuthology.orchestra.run.smithi090.stdout:(18/36): python3-requests-2.25.1-8.el9.noarch.r 383 kB/s | 125 kB 00:00 2024-03-24T12:01:23.952 INFO:teuthology.orchestra.run.smithi006.stdout:(1/12): ceph-grafana-dashboards-19.0.0-1667.gdb 130 kB/s | 31 kB 00:00 2024-03-24T12:01:24.002 INFO:teuthology.orchestra.run.smithi006.stdout:(2/12): ceph-prometheus-alerts-19.0.0-1667.gdb0 52 kB/s | 15 kB 00:00 2024-03-24T12:01:24.086 INFO:teuthology.orchestra.run.smithi006.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.8 MB/s | 238 kB 00:00 2024-03-24T12:01:24.125 INFO:teuthology.orchestra.run.smithi090.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 64 kB/s | 43 kB 00:00 2024-03-24T12:01:24.127 INFO:teuthology.orchestra.run.smithi006.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 5.7 MB/s | 1.0 MB 00:00 2024-03-24T12:01:24.253 INFO:teuthology.orchestra.run.smithi006.stdout:(5/12): ceph-mgr-dashboard-19.0.0-1667.gdb0330b 3.3 MB/s | 1.8 MB 00:00 2024-03-24T12:01:24.292 INFO:teuthology.orchestra.run.smithi090.stdout:(20/36): luarocks-3.9.2-1.el9.noarch.rpm 169 kB/s | 151 kB 00:00 2024-03-24T12:01:24.317 INFO:teuthology.orchestra.run.smithi090.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 212 kB/s | 172 kB 00:00 2024-03-24T12:01:24.328 INFO:teuthology.orchestra.run.smithi006.stdout:(6/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.5 MB/s | 863 kB 00:00 2024-03-24T12:01:24.345 INFO:teuthology.orchestra.run.smithi006.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 211 kB/s | 19 kB 00:00 2024-03-24T12:01:24.370 INFO:teuthology.orchestra.run.smithi006.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 2.2 MB/s | 548 kB 00:00 2024-03-24T12:01:24.404 INFO:teuthology.orchestra.run.smithi006.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 925 kB/s | 31 kB 00:00 2024-03-24T12:01:24.426 INFO:teuthology.orchestra.run.smithi090.stdout:(22/36): python3-jaraco-classes-3.2.1-5.el9.noa 163 kB/s | 18 kB 00:00 2024-03-24T12:01:24.453 INFO:teuthology.orchestra.run.smithi090.stdout:(23/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 67 kB/s | 11 kB 00:00 2024-03-24T12:01:24.462 INFO:teuthology.orchestra.run.smithi006.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.2 MB/s | 188 kB 00:00 2024-03-24T12:01:24.488 INFO:teuthology.orchestra.run.smithi006.stdout:(11/12): python3-grpcio-tools-1.46.7-10.el9.x86 1.0 MB/s | 144 kB 00:00 2024-03-24T12:01:24.561 INFO:teuthology.orchestra.run.smithi090.stdout:(24/36): python3-jaraco-functools-3.5.0-2.el9.n 179 kB/s | 19 kB 00:00 2024-03-24T12:01:24.603 INFO:teuthology.orchestra.run.smithi090.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 131 kB/s | 23 kB 00:00 2024-03-24T12:01:24.671 INFO:teuthology.orchestra.run.smithi090.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 181 kB/s | 20 kB 00:00 2024-03-24T12:01:24.695 INFO:teuthology.orchestra.run.smithi090.stdout:(27/36): python3-cherrypy-18.6.1-2.el9.noarch.r 629 kB/s | 358 kB 00:00 2024-03-24T12:01:24.772 INFO:teuthology.orchestra.run.smithi006.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 4.6 MB/s | 2.0 MB 00:00 2024-03-24T12:01:24.775 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:01:24.775 INFO:teuthology.orchestra.run.smithi006.stdout:Total 5.3 MB/s | 6.9 MB 00:01 2024-03-24T12:01:24.779 INFO:teuthology.orchestra.run.smithi090.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 263 kB/s | 46 kB 00:00 2024-03-24T12:01:24.793 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-24T12:01:24.804 INFO:teuthology.orchestra.run.smithi090.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 588 kB/s | 79 kB 00:00 2024-03-24T12:01:24.812 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-24T12:01:24.812 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-24T12:01:24.913 INFO:teuthology.orchestra.run.smithi090.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.2 MB/s | 272 kB 00:00 2024-03-24T12:01:24.930 INFO:teuthology.orchestra.run.smithi090.stdout:(31/36): python3-portend-3.1.0-2.el9.noarch.rpm 109 kB/s | 16 kB 00:00 2024-03-24T12:01:24.955 INFO:teuthology.orchestra.run.smithi090.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 597 kB/s | 90 kB 00:00 2024-03-24T12:01:24.980 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-24T12:01:24.980 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-24T12:01:25.056 INFO:teuthology.orchestra.run.smithi090.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 251 kB/s | 36 kB 00:00 2024-03-24T12:01:25.198 INFO:teuthology.orchestra.run.smithi090.stdout:(34/36): python3-zc-lockfile-2.0-10.el9.noarch. 141 kB/s | 20 kB 00:00 2024-03-24T12:01:25.223 INFO:teuthology.orchestra.run.smithi090.stdout:(35/36): python3-webob-1.8.7-6.el9.noarch.rpm 785 kB/s | 230 kB 00:00 2024-03-24T12:01:25.273 INFO:teuthology.orchestra.run.smithi090.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.3 MB/s | 427 kB 00:00 2024-03-24T12:01:25.279 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:01:25.279 INFO:teuthology.orchestra.run.smithi090.stdout:Total 8.5 MB/s | 30 MB 00:03 2024-03-24T12:01:25.304 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-03-24T12:01:25.346 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-03-24T12:01:25.346 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-03-24T12:01:25.499 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-24T12:01:25.590 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-03-24T12:01:25.624 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-03-24T12:01:25.671 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-03-24T12:01:25.709 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-03-24T12:01:25.788 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-03-24T12:01:25.788 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-03-24T12:01:25.808 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-03-24T12:01:25.940 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-03-24T12:01:26.040 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-03-24T12:01:26.090 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-03-24T12:01:26.117 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-03-24T12:01:26.149 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1667.gdb0330b1.el9 10/12 2024-03-24T12:01:26.404 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1667.gdb0330b1.el 11/12 2024-03-24T12:01:26.441 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1667.gdb0330b1.el9.noa 12/12 2024-03-24T12:01:27.100 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-03-24T12:01:27.168 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-03-24T12:01:27.200 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-03-24T12:01:27.212 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1667.gdb0330b1.el9.noa 12/12 2024-03-24T12:01:27.212 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1667.gdb0330b1.el 1/12 2024-03-24T12:01:27.212 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1667.gdb0330b1.el9.noa 2/12 2024-03-24T12:01:27.212 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1667.gdb0330b1.el9 3/12 2024-03-24T12:01:27.212 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-03-24T12:01:27.212 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-03-24T12:01:27.212 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-03-24T12:01:27.213 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-03-24T12:01:27.213 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-03-24T12:01:27.213 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-03-24T12:01:27.213 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-03-24T12:01:27.213 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-03-24T12:01:27.229 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-03-24T12:01:27.298 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-03-24T12:01:27.346 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-03-24T12:01:27.387 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-03-24T12:01:27.486 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-03-24T12:01:27.556 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-03-24T12:01:27.562 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-03-24T12:01:27.562 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:27.562 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-24T12:01:27.562 INFO:teuthology.orchestra.run.smithi006.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-24T12:01:27.562 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-grafana-dashboards-2:19.0.0-1667.gdb0330b1.el9.noarch 2024-03-24T12:01:27.562 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-dashboard-2:19.0.0-1667.gdb0330b1.el9.noarch 2024-03-24T12:01:27.562 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-prometheus-alerts-2:19.0.0-1667.gdb0330b1.el9.noarch 2024-03-24T12:01:27.562 INFO:teuthology.orchestra.run.smithi006.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-24T12:01:27.562 INFO:teuthology.orchestra.run.smithi006.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-24T12:01:27.563 INFO:teuthology.orchestra.run.smithi006.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-24T12:01:27.563 INFO:teuthology.orchestra.run.smithi006.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-24T12:01:27.563 INFO:teuthology.orchestra.run.smithi006.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-24T12:01:27.563 INFO:teuthology.orchestra.run.smithi006.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-24T12:01:27.563 INFO:teuthology.orchestra.run.smithi006.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-24T12:01:27.563 INFO:teuthology.orchestra.run.smithi006.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-24T12:01:27.563 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:27.563 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:01:27.598 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-03-24T12:01:27.669 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-03-24T12:01:27.724 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-03-24T12:01:27.763 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-03-24T12:01:27.820 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-03-24T12:01:27.821 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-24T12:01:27.879 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-03-24T12:01:28.073 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-03-24T12:01:28.127 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-03-24T12:01:28.169 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-03-24T12:01:28.209 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-03-24T12:01:28.239 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-03-24T12:01:28.314 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-03-24T12:01:28.350 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-24T12:01:28.383 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-03-24T12:01:28.390 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:14 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:01:28.423 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-03-24T12:01:28.467 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-03-24T12:01:28.503 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-03-24T12:01:28.549 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:01:28.550 INFO:teuthology.orchestra.run.smithi006.stdout:============================================================================================ 2024-03-24T12:01:28.550 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-03-24T12:01:28.550 INFO:teuthology.orchestra.run.smithi006.stdout:============================================================================================ 2024-03-24T12:01:28.550 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-24T12:01:28.550 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1667.gdb0330b1.el9 ceph-noarch 7.4 M 2024-03-24T12:01:28.550 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-03-24T12:01:28.551 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-24T12:01:28.551 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-24T12:01:28.551 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-24T12:01:28.551 INFO:teuthology.orchestra.run.smithi006.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-24T12:01:28.551 INFO:teuthology.orchestra.run.smithi006.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-24T12:01:28.551 INFO:teuthology.orchestra.run.smithi006.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-24T12:01:28.551 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-24T12:01:28.551 INFO:teuthology.orchestra.run.smithi006.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-24T12:01:28.551 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-24T12:01:28.551 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-24T12:01:28.551 INFO:teuthology.orchestra.run.smithi006.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-24T12:01:28.551 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:28.551 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-24T12:01:28.551 INFO:teuthology.orchestra.run.smithi006.stdout:============================================================================================ 2024-03-24T12:01:28.551 INFO:teuthology.orchestra.run.smithi006.stdout:Install 12 Packages 2024-03-24T12:01:28.552 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:28.552 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 37 M 2024-03-24T12:01:28.552 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 209 M 2024-03-24T12:01:28.552 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-24T12:01:28.591 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-03-24T12:01:28.626 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-03-24T12:01:28.697 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-03-24T12:01:28.811 INFO:teuthology.orchestra.run.smithi006.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 227 kB/s | 32 kB 00:00 2024-03-24T12:01:28.833 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-03-24T12:01:28.845 INFO:teuthology.orchestra.run.smithi006.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 537 kB/s | 17 kB 00:00 2024-03-24T12:01:28.878 INFO:teuthology.orchestra.run.smithi006.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2024-03-24T12:01:28.945 INFO:teuthology.orchestra.run.smithi006.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2024-03-24T12:01:28.950 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-24T12:01:29.111 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1667.gdb0330b1.el9. 31/36 2024-03-24T12:01:29.121 INFO:teuthology.orchestra.run.smithi006.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2024-03-24T12:01:29.133 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-2:19.0.0-1667.gdb0330b1.el9.x86_64 32/36 2024-03-24T12:01:29.159 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1667.gdb0330b1.el9.x86_64 32/36 2024-03-24T12:01:29.159 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T12:01:29.159 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-24T12:01:29.159 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-24T12:01:29.160 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-24T12:01:29.160 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:29.187 INFO:teuthology.orchestra.run.smithi006.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 4.8 MB 00:00 2024-03-24T12:01:29.213 INFO:teuthology.orchestra.run.smithi006.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.4 MB/s | 217 kB 00:00 2024-03-24T12:01:29.513 INFO:teuthology.orchestra.run.smithi006.stdout:(8/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 48 MB/s | 16 MB 00:00 2024-03-24T12:01:29.672 INFO:teuthology.orchestra.run.smithi006.stdout:(9/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-03-24T12:01:29.706 INFO:teuthology.orchestra.run.smithi006.stdout:(10/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 803 kB 00:00 2024-03-24T12:01:30.264 INFO:teuthology.orchestra.run.smithi006.stdout:(11/12): ceph-mgr-diskprediction-local-19.0.0-1 4.6 MB/s | 7.4 MB 00:01 2024-03-24T12:01:30.289 INFO:teuthology.orchestra.run.smithi006.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 397 kB/s | 245 kB 00:00 2024-03-24T12:01:30.291 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:01:30.292 INFO:teuthology.orchestra.run.smithi006.stdout:Total 21 MB/s | 37 MB 00:01 2024-03-24T12:01:30.334 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-24T12:01:30.359 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-24T12:01:30.359 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-24T12:01:30.593 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-osd-2:19.0.0-1667.gdb0330b1.el9.x86_64 33/36 2024-03-24T12:01:30.619 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-osd-2:19.0.0-1667.gdb0330b1.el9.x86_64 33/36 2024-03-24T12:01:30.619 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T12:01:30.619 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-24T12:01:30.619 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-24T12:01:30.619 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-24T12:01:30.619 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:30.729 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-24T12:01:30.729 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-24T12:01:31.069 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mon-2:19.0.0-1667.gdb0330b1.el9.x86_64 34/36 2024-03-24T12:01:31.092 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mon-2:19.0.0-1667.gdb0330b1.el9.x86_64 34/36 2024-03-24T12:01:31.092 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T12:01:31.092 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-24T12:01:31.092 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-24T12:01:31.092 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-24T12:01:31.092 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:31.303 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mds-2:19.0.0-1667.gdb0330b1.el9.x86_64 35/36 2024-03-24T12:01:31.325 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mds-2:19.0.0-1667.gdb0330b1.el9.x86_64 35/36 2024-03-24T12:01:31.325 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T12:01:31.326 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-24T12:01:31.326 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-24T12:01:31.326 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-24T12:01:31.326 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:31.391 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-2:19.0.0-1667.gdb0330b1.el9.x86_64 36/36 2024-03-24T12:01:31.450 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-24T12:01:31.530 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-24T12:01:31.560 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-24T12:01:31.659 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-24T12:01:31.706 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-24T12:01:31.890 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-24T12:01:31.921 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-24T12:01:32.009 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-24T12:01:32.707 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-24T12:01:32.812 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-24T12:01:33.306 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-2:19.0.0-1667.gdb0330b1.el9.x86_64 36/36 2024-03-24T12:01:33.306 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/36 2024-03-24T12:01:33.306 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mds-2:19.0.0-1667.gdb0330b1.el9.x86_64 2/36 2024-03-24T12:01:33.306 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-2:19.0.0-1667.gdb0330b1.el9.x86_64 3/36 2024-03-24T12:01:33.306 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mon-2:19.0.0-1667.gdb0330b1.el9.x86_64 4/36 2024-03-24T12:01:33.306 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-osd-2:19.0.0-1667.gdb0330b1.el9.x86_64 5/36 2024-03-24T12:01:33.306 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1667.gdb0330b1.el9. 6/36 2024-03-24T12:01:33.306 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-03-24T12:01:33.306 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-03-24T12:01:33.306 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-03-24T12:01:33.306 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-03-24T12:01:33.307 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-03-24T12:01:33.307 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-03-24T12:01:33.307 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-03-24T12:01:33.307 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-03-24T12:01:33.307 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-03-24T12:01:33.307 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/36 2024-03-24T12:01:33.307 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 17/36 2024-03-24T12:01:33.307 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-03-24T12:01:33.307 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-03-24T12:01:33.307 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-03-24T12:01:33.307 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-24T12:01:33.307 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-03-24T12:01:33.307 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-03-24T12:01:33.307 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-03-24T12:01:33.307 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-03-24T12:01:33.308 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-03-24T12:01:33.308 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-03-24T12:01:33.309 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-03-24T12:01:33.309 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-03-24T12:01:33.309 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-24T12:01:33.309 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-03-24T12:01:33.309 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-03-24T12:01:33.309 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-03-24T12:01:33.309 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-03-24T12:01:33.309 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-03-24T12:01:34.121 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-24T12:01:34.382 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-03-24T12:01:34.382 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:34.382 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-03-24T12:01:34.383 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:34.383 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:34.383 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:34.383 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core-2:19.0.0-1667.gdb0330b1.el9.noarch 2024-03-24T12:01:34.383 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:34.383 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:34.383 INFO:teuthology.orchestra.run.smithi090.stdout: lua-5.4.4-4.el9.x86_64 2024-03-24T12:01:34.383 INFO:teuthology.orchestra.run.smithi090.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-24T12:01:34.383 INFO:teuthology.orchestra.run.smithi090.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-24T12:01:34.383 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-24T12:01:34.383 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-24T12:01:34.383 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-24T12:01:34.383 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-24T12:01:34.383 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-24T12:01:34.384 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-24T12:01:34.384 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-24T12:01:34.384 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-24T12:01:34.384 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-24T12:01:34.384 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-24T12:01:34.384 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-24T12:01:34.384 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-24T12:01:34.384 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-24T12:01:34.384 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-24T12:01:34.384 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-24T12:01:34.384 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply-3.11-14.el9.noarch 2024-03-24T12:01:34.384 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-24T12:01:34.384 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-24T12:01:34.384 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-24T12:01:34.384 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-24T12:01:34.385 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-24T12:01:34.385 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-24T12:01:34.385 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-24T12:01:34.385 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-24T12:01:34.385 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-24T12:01:34.385 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-24T12:01:34.385 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-24T12:01:34.385 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:34.385 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:01:34.935 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-base 2024-03-24T12:01:35.197 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-24T12:01:35.220 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1667.gdb033 12/12 2024-03-24T12:01:35.500 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:57 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:01:35.596 INFO:teuthology.orchestra.run.smithi090.stdout:Package ceph-base-2:19.0.0-1667.gdb0330b1.el9.x86_64 is already installed. 2024-03-24T12:01:35.648 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:01:35.649 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-03-24T12:01:35.649 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:01:35.726 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install cephadm 2024-03-24T12:01:36.283 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:58 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:01:36.429 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:01:36.429 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:01:36.429 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-03-24T12:01:36.429 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:01:36.429 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-03-24T12:01:36.429 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm noarch 2:19.0.0-1667.gdb0330b1.el9 ceph-noarch 748 k 2024-03-24T12:01:36.429 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:36.429 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-03-24T12:01:36.430 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:01:36.430 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-03-24T12:01:36.430 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:36.430 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 748 k 2024-03-24T12:01:36.430 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 754 k 2024-03-24T12:01:36.430 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-03-24T12:01:36.733 INFO:teuthology.orchestra.run.smithi090.stdout:cephadm-19.0.0-1667.gdb0330b1.el9.noarch.rpm 2.4 MB/s | 748 kB 00:00 2024-03-24T12:01:36.734 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:01:36.734 INFO:teuthology.orchestra.run.smithi090.stdout:Total 2.4 MB/s | 748 kB 00:00 2024-03-24T12:01:36.734 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-03-24T12:01:36.739 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-03-24T12:01:36.739 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-03-24T12:01:36.745 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-03-24T12:01:36.745 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-03-24T12:01:36.820 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-03-24T12:01:37.226 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephadm-2:19.0.0-1667.gdb0330b1.el9.noarch 1/1 2024-03-24T12:01:37.314 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephadm-2:19.0.0-1667.gdb0330b1.el9.noarch 1/1 2024-03-24T12:01:37.751 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephadm-2:19.0.0-1667.gdb0330b1.el9.noarch 1/1 2024-03-24T12:01:37.782 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1667.gdb033 12/12 2024-03-24T12:01:37.782 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1667.gdb033 1/12 2024-03-24T12:01:37.782 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-24T12:01:37.782 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-24T12:01:37.782 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-24T12:01:37.782 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-24T12:01:37.782 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-24T12:01:37.782 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-24T12:01:37.782 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-24T12:01:37.783 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-24T12:01:37.783 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-24T12:01:37.783 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-24T12:01:38.044 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephadm-2:19.0.0-1667.gdb0330b1.el9.noarch 1/1 2024-03-24T12:01:38.044 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:38.044 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-03-24T12:01:38.044 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm-2:19.0.0-1667.gdb0330b1.el9.noarch 2024-03-24T12:01:38.044 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:38.044 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:01:38.204 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-24T12:01:38.204 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:38.204 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-24T12:01:38.205 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1667.gdb0330b1.el9.noarch 2024-03-24T12:01:38.205 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-24T12:01:38.205 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-24T12:01:38.205 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-24T12:01:38.205 INFO:teuthology.orchestra.run.smithi006.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-24T12:01:38.205 INFO:teuthology.orchestra.run.smithi006.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-24T12:01:38.205 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-24T12:01:38.205 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-24T12:01:38.205 INFO:teuthology.orchestra.run.smithi006.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-24T12:01:38.205 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-24T12:01:38.205 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-24T12:01:38.205 INFO:teuthology.orchestra.run.smithi006.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-24T12:01:38.205 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:38.205 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:01:38.300 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-immutable-object-cache 2024-03-24T12:01:38.609 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-mgr-rook 2024-03-24T12:01:38.867 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:00 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:01:39.023 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:01:39.023 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:01:39.024 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repo Size 2024-03-24T12:01:39.024 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:01:39.024 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-03-24T12:01:39.024 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 145 k 2024-03-24T12:01:39.024 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:39.024 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-03-24T12:01:39.024 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:01:39.024 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-03-24T12:01:39.024 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:39.024 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 145 k 2024-03-24T12:01:39.024 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 436 k 2024-03-24T12:01:39.024 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-03-24T12:01:39.186 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:25 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:01:39.253 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-immutable-object-cache-19.0.0-1667.gdb0330 636 kB/s | 145 kB 00:00 2024-03-24T12:01:39.254 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:01:39.254 INFO:teuthology.orchestra.run.smithi090.stdout:Total 632 kB/s | 145 kB 00:00 2024-03-24T12:01:39.254 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-03-24T12:01:39.262 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-03-24T12:01:39.262 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-03-24T12:01:39.313 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-03-24T12:01:39.314 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-03-24T12:01:39.344 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:01:39.345 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================== 2024-03-24T12:01:39.345 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-03-24T12:01:39.345 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================== 2024-03-24T12:01:39.346 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-24T12:01:39.346 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-rook noarch 2:19.0.0-1667.gdb0330b1.el9 ceph-noarch 49 k 2024-03-24T12:01:39.346 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-03-24T12:01:39.346 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-24T12:01:39.346 INFO:teuthology.orchestra.run.smithi006.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-24T12:01:39.346 INFO:teuthology.orchestra.run.smithi006.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-24T12:01:39.346 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-24T12:01:39.346 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-24T12:01:39.346 INFO:teuthology.orchestra.run.smithi006.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-24T12:01:39.346 INFO:teuthology.orchestra.run.smithi006.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-24T12:01:39.346 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-24T12:01:39.346 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-24T12:01:39.346 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-24T12:01:39.346 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-24T12:01:39.347 INFO:teuthology.orchestra.run.smithi006.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-24T12:01:39.347 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:39.347 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-24T12:01:39.347 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================== 2024-03-24T12:01:39.347 INFO:teuthology.orchestra.run.smithi006.stdout:Install 13 Packages 2024-03-24T12:01:39.347 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:39.347 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 2.0 M 2024-03-24T12:01:39.348 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 26 M 2024-03-24T12:01:39.348 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-24T12:01:39.450 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-03-24T12:01:39.521 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1667.gdb0330b1. 1/1 2024-03-24T12:01:39.545 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1667.gdb0330b1. 1/1 2024-03-24T12:01:39.545 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T12:01:39.545 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-24T12:01:39.545 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:39.638 INFO:teuthology.orchestra.run.smithi006.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 183 kB/s | 24 kB 00:00 2024-03-24T12:01:39.663 INFO:teuthology.orchestra.run.smithi006.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 114 kB/s | 18 kB 00:00 2024-03-24T12:01:39.713 INFO:teuthology.orchestra.run.smithi006.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-03-24T12:01:39.738 INFO:teuthology.orchestra.run.smithi006.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-24T12:01:39.772 INFO:teuthology.orchestra.run.smithi006.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.5 MB/s | 211 kB 00:00 2024-03-24T12:01:39.789 INFO:teuthology.orchestra.run.smithi006.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 851 kB/s | 43 kB 00:00 2024-03-24T12:01:39.806 INFO:teuthology.orchestra.run.smithi006.stdout:(7/13): ceph-mgr-rook-19.0.0-1667.gdb0330b1.el9 163 kB/s | 49 kB 00:00 2024-03-24T12:01:39.957 INFO:teuthology.orchestra.run.smithi006.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 175 kB/s | 32 kB 00:00 2024-03-24T12:01:39.974 INFO:teuthology.orchestra.run.smithi006.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 81 kB/s | 15 kB 00:00 2024-03-24T12:01:40.049 INFO:teuthology.orchestra.run.smithi006.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 789 kB/s | 59 kB 00:00 2024-03-24T12:01:40.090 INFO:teuthology.orchestra.run.smithi006.stdout:(11/13): python3-google-auth-2.28.2-1.el9.noarc 865 kB/s | 203 kB 00:00 2024-03-24T12:01:40.116 INFO:teuthology.orchestra.run.smithi006.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.3 MB/s | 90 kB 00:00 2024-03-24T12:01:40.416 INFO:teuthology.orchestra.run.smithi006.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.2 MB/s | 1.0 MB 00:00 2024-03-24T12:01:40.420 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:01:40.421 INFO:teuthology.orchestra.run.smithi006.stdout:Total 1.9 MB/s | 2.0 MB 00:01 2024-03-24T12:01:40.421 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-24T12:01:40.435 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-24T12:01:40.435 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-24T12:01:40.530 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-24T12:01:40.530 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-24T12:01:40.619 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1667.gdb0330b1. 1/1 2024-03-24T12:01:40.619 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:40.619 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-03-24T12:01:40.619 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:40.619 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:40.619 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:01:40.910 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr 2024-03-24T12:01:41.051 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-24T12:01:41.221 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-24T12:01:41.277 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-24T12:01:41.341 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-24T12:01:41.377 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-24T12:01:41.419 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-24T12:01:41.490 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:03 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:01:41.516 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-24T12:01:41.585 INFO:teuthology.orchestra.run.smithi090.stdout:Package ceph-mgr-2:19.0.0-1667.gdb0330b1.el9.x86_64 is already installed. 2024-03-24T12:01:41.626 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-24T12:01:41.636 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:01:41.637 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-03-24T12:01:41.637 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:01:41.679 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-24T12:01:41.705 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr-dashboard 2024-03-24T12:01:42.264 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:04 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:01:42.414 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:01:42.415 INFO:teuthology.orchestra.run.smithi090.stdout:====================================================================================== 2024-03-24T12:01:42.415 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-03-24T12:01:42.416 INFO:teuthology.orchestra.run.smithi090.stdout:====================================================================================== 2024-03-24T12:01:42.416 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-03-24T12:01:42.416 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1667.gdb0330b1.el9 ceph-noarch 1.8 M 2024-03-24T12:01:42.416 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-03-24T12:01:42.416 INFO:teuthology.orchestra.run.smithi090.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-24T12:01:42.416 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1667.gdb0330b1.el9 ceph-noarch 31 k 2024-03-24T12:01:42.416 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1667.gdb0330b1.el9 ceph-noarch 15 k 2024-03-24T12:01:42.416 INFO:teuthology.orchestra.run.smithi090.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-24T12:01:42.417 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-24T12:01:42.417 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-24T12:01:42.417 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-24T12:01:42.417 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-24T12:01:42.417 INFO:teuthology.orchestra.run.smithi090.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-24T12:01:42.417 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-24T12:01:42.417 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-24T12:01:42.417 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:42.417 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-03-24T12:01:42.417 INFO:teuthology.orchestra.run.smithi090.stdout:====================================================================================== 2024-03-24T12:01:42.418 INFO:teuthology.orchestra.run.smithi090.stdout:Install 12 Packages 2024-03-24T12:01:42.418 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:42.418 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 6.9 M 2024-03-24T12:01:42.418 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 26 M 2024-03-24T12:01:42.419 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-03-24T12:01:42.442 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-24T12:01:42.483 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-24T12:01:42.517 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-24T12:01:42.566 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-24T12:01:42.595 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-rook-2:19.0.0-1667.gdb0330b1.el9.noarch 13/13 2024-03-24T12:01:43.029 INFO:teuthology.orchestra.run.smithi090.stdout:(1/12): ceph-grafana-dashboards-19.0.0-1667.gdb 120 kB/s | 31 kB 00:00 2024-03-24T12:01:43.054 INFO:teuthology.orchestra.run.smithi090.stdout:(2/12): ceph-prometheus-alerts-19.0.0-1667.gdb0 54 kB/s | 15 kB 00:00 2024-03-24T12:01:43.180 INFO:teuthology.orchestra.run.smithi090.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.9 MB/s | 238 kB 00:00 2024-03-24T12:01:43.238 INFO:teuthology.orchestra.run.smithi090.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.8 MB/s | 1.0 MB 00:00 2024-03-24T12:01:43.355 INFO:teuthology.orchestra.run.smithi090.stdout:(5/12): ceph-mgr-dashboard-19.0.0-1667.gdb0330b 3.0 MB/s | 1.8 MB 00:00 2024-03-24T12:01:43.410 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1667.gdb0330b1.el9.noarch 13/13 2024-03-24T12:01:43.410 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1667.gdb0330b1.el9.noarch 1/13 2024-03-24T12:01:43.410 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-24T12:01:43.410 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-24T12:01:43.410 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-24T12:01:43.410 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-24T12:01:43.411 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-24T12:01:43.411 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-24T12:01:43.411 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-24T12:01:43.411 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-24T12:01:43.411 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-24T12:01:43.411 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-24T12:01:43.411 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-24T12:01:43.430 INFO:teuthology.orchestra.run.smithi090.stdout:(6/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.4 MB/s | 863 kB 00:00 2024-03-24T12:01:43.648 INFO:teuthology.orchestra.run.smithi090.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 66 kB/s | 19 kB 00:00 2024-03-24T12:01:43.714 INFO:teuthology.orchestra.run.smithi090.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.1 MB/s | 548 kB 00:00 2024-03-24T12:01:43.779 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-24T12:01:43.779 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:43.779 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-24T12:01:43.779 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-rook-2:19.0.0-1667.gdb0330b1.el9.noarch 2024-03-24T12:01:43.779 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-24T12:01:43.779 INFO:teuthology.orchestra.run.smithi006.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-24T12:01:43.779 INFO:teuthology.orchestra.run.smithi006.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-24T12:01:43.779 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-24T12:01:43.779 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-24T12:01:43.779 INFO:teuthology.orchestra.run.smithi006.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-24T12:01:43.779 INFO:teuthology.orchestra.run.smithi006.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-24T12:01:43.780 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-24T12:01:43.780 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-24T12:01:43.780 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-24T12:01:43.780 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-24T12:01:43.780 INFO:teuthology.orchestra.run.smithi006.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-24T12:01:43.780 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:43.780 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:01:43.782 INFO:teuthology.orchestra.run.smithi090.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 461 kB/s | 31 kB 00:00 2024-03-24T12:01:43.840 INFO:teuthology.orchestra.run.smithi090.stdout:(10/12): python3-grpcio-tools-1.46.7-10.el9.x86 751 kB/s | 144 kB 00:00 2024-03-24T12:01:43.865 INFO:teuthology.orchestra.run.smithi090.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.2 MB/s | 188 kB 00:00 2024-03-24T12:01:44.009 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-mgr-cephadm 2024-03-24T12:01:44.049 INFO:teuthology.orchestra.run.smithi090.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 3.4 MB/s | 2.0 MB 00:00 2024-03-24T12:01:44.054 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:01:44.054 INFO:teuthology.orchestra.run.smithi090.stdout:Total 4.2 MB/s | 6.9 MB 00:01 2024-03-24T12:01:44.073 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-03-24T12:01:44.090 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-03-24T12:01:44.090 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-03-24T12:01:44.254 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-03-24T12:01:44.254 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-03-24T12:01:44.577 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:30 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:01:44.733 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:01:44.734 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================== 2024-03-24T12:01:44.734 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-03-24T12:01:44.734 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================== 2024-03-24T12:01:44.734 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-24T12:01:44.734 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1667.gdb0330b1.el9 ceph-noarch 138 k 2024-03-24T12:01:44.734 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-03-24T12:01:44.734 INFO:teuthology.orchestra.run.smithi006.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-24T12:01:44.734 INFO:teuthology.orchestra.run.smithi006.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-24T12:01:44.734 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-24T12:01:44.734 INFO:teuthology.orchestra.run.smithi006.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-24T12:01:44.734 INFO:teuthology.orchestra.run.smithi006.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-24T12:01:44.734 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:44.735 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-24T12:01:44.735 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================== 2024-03-24T12:01:44.735 INFO:teuthology.orchestra.run.smithi006.stdout:Install 6 Packages 2024-03-24T12:01:44.735 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:44.735 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 6.8 M 2024-03-24T12:01:44.735 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 33 M 2024-03-24T12:01:44.735 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-24T12:01:44.770 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-03-24T12:01:44.850 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-03-24T12:01:44.895 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-03-24T12:01:44.942 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-03-24T12:01:44.981 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-03-24T12:01:45.082 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-03-24T12:01:45.216 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-03-24T12:01:45.311 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-03-24T12:01:45.363 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-03-24T12:01:45.400 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-03-24T12:01:45.440 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1667.gdb0330b1.el9 10/12 2024-03-24T12:01:45.682 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1667.gdb0330b1.el 11/12 2024-03-24T12:01:45.709 INFO:teuthology.orchestra.run.smithi006.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.9 MB/s | 248 kB 00:00 2024-03-24T12:01:45.715 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1667.gdb0330b1.el9.noa 12/12 2024-03-24T12:01:45.876 INFO:teuthology.orchestra.run.smithi006.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 5.8 MB 00:00 2024-03-24T12:01:45.977 INFO:teuthology.orchestra.run.smithi006.stdout:(3/6): ceph-mgr-cephadm-19.0.0-1667.gdb0330b1.e 352 kB/s | 138 kB 00:00 2024-03-24T12:01:46.085 INFO:teuthology.orchestra.run.smithi006.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 278 kB/s | 58 kB 00:00 2024-03-24T12:01:46.102 INFO:teuthology.orchestra.run.smithi006.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 407 kB/s | 51 kB 00:00 2024-03-24T12:01:46.177 INFO:teuthology.orchestra.run.smithi006.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.1 MB/s | 548 kB 00:00 2024-03-24T12:01:46.180 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:01:46.180 INFO:teuthology.orchestra.run.smithi006.stdout:Total 4.7 MB/s | 6.8 MB 00:01 2024-03-24T12:01:46.193 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-24T12:01:46.205 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-24T12:01:46.205 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-24T12:01:46.298 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-24T12:01:46.298 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-24T12:01:46.621 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-24T12:01:46.674 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1667.gdb0330b1.el9.noa 12/12 2024-03-24T12:01:46.675 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1667.gdb0330b1.el 1/12 2024-03-24T12:01:46.675 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1667.gdb0330b1.el9.noa 2/12 2024-03-24T12:01:46.675 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1667.gdb0330b1.el9 3/12 2024-03-24T12:01:46.675 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-03-24T12:01:46.675 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-03-24T12:01:46.675 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-03-24T12:01:46.675 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-03-24T12:01:46.675 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-03-24T12:01:46.675 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-03-24T12:01:46.675 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-03-24T12:01:46.675 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-03-24T12:01:46.791 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-24T12:01:46.838 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-24T12:01:47.016 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-03-24T12:01:47.016 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:47.016 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-03-24T12:01:47.016 INFO:teuthology.orchestra.run.smithi090.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-24T12:01:47.016 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards-2:19.0.0-1667.gdb0330b1.el9.noarch 2024-03-24T12:01:47.017 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard-2:19.0.0-1667.gdb0330b1.el9.noarch 2024-03-24T12:01:47.017 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts-2:19.0.0-1667.gdb0330b1.el9.noarch 2024-03-24T12:01:47.017 INFO:teuthology.orchestra.run.smithi090.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-24T12:01:47.017 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-24T12:01:47.017 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-24T12:01:47.017 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-24T12:01:47.017 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-24T12:01:47.017 INFO:teuthology.orchestra.run.smithi090.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-24T12:01:47.017 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-24T12:01:47.017 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-24T12:01:47.017 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:47.017 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:01:47.334 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-24T12:01:47.397 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-24T12:01:47.475 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-24T12:01:47.536 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-24T12:01:47.566 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1667.gdb0330b1.el9.noarch 6/6 2024-03-24T12:01:47.909 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:09 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:01:48.069 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:01:48.070 INFO:teuthology.orchestra.run.smithi090.stdout:============================================================================================ 2024-03-24T12:01:48.070 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-03-24T12:01:48.070 INFO:teuthology.orchestra.run.smithi090.stdout:============================================================================================ 2024-03-24T12:01:48.070 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-03-24T12:01:48.070 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1667.gdb0330b1.el9 ceph-noarch 7.4 M 2024-03-24T12:01:48.070 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-03-24T12:01:48.071 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-24T12:01:48.071 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-24T12:01:48.071 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-24T12:01:48.071 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-24T12:01:48.071 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-24T12:01:48.071 INFO:teuthology.orchestra.run.smithi090.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-24T12:01:48.071 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-24T12:01:48.071 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-24T12:01:48.071 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-24T12:01:48.071 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-24T12:01:48.071 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-24T12:01:48.071 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:48.071 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-03-24T12:01:48.071 INFO:teuthology.orchestra.run.smithi090.stdout:============================================================================================ 2024-03-24T12:01:48.071 INFO:teuthology.orchestra.run.smithi090.stdout:Install 12 Packages 2024-03-24T12:01:48.072 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:48.072 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 37 M 2024-03-24T12:01:48.072 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 209 M 2024-03-24T12:01:48.072 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-03-24T12:01:48.377 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1667.gdb0330b1.el9.noarch 6/6 2024-03-24T12:01:48.378 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1667.gdb0330b1.el9.noarch 1/6 2024-03-24T12:01:48.378 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-24T12:01:48.378 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-24T12:01:48.378 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-24T12:01:48.378 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-24T12:01:48.678 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-24T12:01:48.678 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:48.678 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-24T12:01:48.678 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-cephadm-2:19.0.0-1667.gdb0330b1.el9.noarch 2024-03-24T12:01:48.678 INFO:teuthology.orchestra.run.smithi006.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-24T12:01:48.678 INFO:teuthology.orchestra.run.smithi006.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-24T12:01:48.679 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-24T12:01:48.679 INFO:teuthology.orchestra.run.smithi006.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-24T12:01:48.679 INFO:teuthology.orchestra.run.smithi006.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-24T12:01:48.679 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:48.679 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:01:48.764 INFO:teuthology.orchestra.run.smithi090.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 235 kB/s | 32 kB 00:00 2024-03-24T12:01:48.798 INFO:teuthology.orchestra.run.smithi090.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 533 kB/s | 17 kB 00:00 2024-03-24T12:01:48.831 INFO:teuthology.orchestra.run.smithi090.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2024-03-24T12:01:48.891 INFO:teuthology.orchestra.run.smithi090.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2024-03-24T12:01:48.929 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-fuse 2024-03-24T12:01:49.124 INFO:teuthology.orchestra.run.smithi090.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2024-03-24T12:01:49.200 INFO:teuthology.orchestra.run.smithi090.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 4.8 MB 00:00 2024-03-24T12:01:49.233 INFO:teuthology.orchestra.run.smithi090.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.0 MB/s | 217 kB 00:00 2024-03-24T12:01:49.498 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:35 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:01:49.592 INFO:teuthology.orchestra.run.smithi090.stdout:(8/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 40 MB/s | 16 MB 00:00 2024-03-24T12:01:49.653 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:01:49.653 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:01:49.654 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-03-24T12:01:49.654 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:01:49.654 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-24T12:01:49.654 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-fuse x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 886 k 2024-03-24T12:01:49.654 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-03-24T12:01:49.654 INFO:teuthology.orchestra.run.smithi006.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-24T12:01:49.654 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:49.654 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-24T12:01:49.654 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:01:49.654 INFO:teuthology.orchestra.run.smithi006.stdout:Install 2 Packages 2024-03-24T12:01:49.654 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:49.655 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 964 k 2024-03-24T12:01:49.655 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 2.8 M 2024-03-24T12:01:49.655 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-24T12:01:49.787 INFO:teuthology.orchestra.run.smithi006.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 596 kB/s | 79 kB 00:00 2024-03-24T12:01:49.875 INFO:teuthology.orchestra.run.smithi090.stdout:(9/12): libquadmath-11.4.1-3.el9.x86_64.rpm 690 kB/s | 195 kB 00:00 2024-03-24T12:01:50.151 INFO:teuthology.orchestra.run.smithi090.stdout:(10/12): ceph-mgr-diskprediction-local-19.0.0-1 4.8 MB/s | 7.4 MB 00:01 2024-03-24T12:01:50.170 INFO:teuthology.orchestra.run.smithi006.stdout:(2/2): ceph-fuse-19.0.0-1667.gdb0330b1.el9.x86_ 1.7 MB/s | 886 kB 00:00 2024-03-24T12:01:50.172 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:01:50.172 INFO:teuthology.orchestra.run.smithi006.stdout:Total 1.8 MB/s | 964 kB 00:00 2024-03-24T12:01:50.172 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-24T12:01:50.192 INFO:teuthology.orchestra.run.smithi090.stdout:(11/12): libgfortran-11.4.1-3.el9.x86_64.rpm 839 kB/s | 803 kB 00:00 2024-03-24T12:01:50.194 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-24T12:01:50.194 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-24T12:01:50.218 INFO:teuthology.orchestra.run.smithi090.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 717 kB/s | 245 kB 00:00 2024-03-24T12:01:50.220 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:01:50.220 INFO:teuthology.orchestra.run.smithi090.stdout:Total 17 MB/s | 37 MB 00:02 2024-03-24T12:01:50.260 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-03-24T12:01:50.271 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-24T12:01:50.271 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-24T12:01:50.284 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-03-24T12:01:50.285 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-03-24T12:01:50.461 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-24T12:01:50.626 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-24T12:01:50.644 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-03-24T12:01:50.644 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-03-24T12:01:50.698 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-fuse-2:19.0.0-1667.gdb0330b1.el9.x86_64 2/2 2024-03-24T12:01:51.346 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-03-24T12:01:51.384 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-fuse-2:19.0.0-1667.gdb0330b1.el9.x86_64 2/2 2024-03-24T12:01:51.384 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-fuse-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/2 2024-03-24T12:01:51.433 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-24T12:01:51.464 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-24T12:01:51.575 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-24T12:01:51.611 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-24T12:01:51.709 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-24T12:01:51.709 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:51.709 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-24T12:01:51.709 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-fuse-2:19.0.0-1667.gdb0330b1.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-24T12:01:51.709 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:51.710 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:01:51.789 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-24T12:01:51.818 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-24T12:01:51.911 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-24T12:01:51.992 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-volume 2024-03-24T12:01:52.568 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:38 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:01:52.578 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-24T12:01:52.692 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-24T12:01:52.725 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:01:52.726 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:01:52.726 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2024-03-24T12:01:52.726 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:01:52.726 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-24T12:01:52.726 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-volume noarch 2:19.0.0-1667.gdb0330b1.el9 ceph-noarch 264 k 2024-03-24T12:01:52.726 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2024-03-24T12:01:52.726 INFO:teuthology.orchestra.run.smithi006.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-24T12:01:52.726 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:52.726 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-24T12:01:52.726 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:01:52.726 INFO:teuthology.orchestra.run.smithi006.stdout:Install 2 Packages 2024-03-24T12:01:52.727 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:52.727 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 332 k 2024-03-24T12:01:52.727 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 1.5 M 2024-03-24T12:01:52.727 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-24T12:01:52.850 INFO:teuthology.orchestra.run.smithi006.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 560 kB/s | 69 kB 00:00 2024-03-24T12:01:53.009 INFO:teuthology.orchestra.run.smithi006.stdout:(2/2): ceph-volume-19.0.0-1667.gdb0330b1.el9.no 938 kB/s | 264 kB 00:00 2024-03-24T12:01:53.010 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:01:53.010 INFO:teuthology.orchestra.run.smithi006.stdout:Total 1.1 MB/s | 332 kB 00:00 2024-03-24T12:01:53.011 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-24T12:01:53.020 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-24T12:01:53.020 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-24T12:01:53.055 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-24T12:01:53.056 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-24T12:01:53.210 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-24T12:01:53.397 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-03-24T12:01:53.428 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-volume-2:19.0.0-1667.gdb0330b1.el9.noarch 2/2 2024-03-24T12:01:53.443 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-volume-2:19.0.0-1667.gdb0330b1.el9.noarch 2/2 2024-03-24T12:01:53.443 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T12:01:53.443 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-24T12:01:53.443 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:53.970 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-24T12:01:54.170 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-volume-2:19.0.0-1667.gdb0330b1.el9.noarch 1/2 2024-03-24T12:01:54.484 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-03-24T12:01:54.484 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:54.484 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-24T12:01:54.484 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-volume-2:19.0.0-1667.gdb0330b1.el9.noarch 2024-03-24T12:01:54.484 INFO:teuthology.orchestra.run.smithi006.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-24T12:01:54.484 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:54.484 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:01:54.872 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install librados-devel 2024-03-24T12:01:54.999 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-24T12:01:55.034 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1667.gdb033 12/12 2024-03-24T12:01:55.448 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:41 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:01:55.605 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:01:55.605 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:01:55.606 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repo Size 2024-03-24T12:01:55.606 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:01:55.606 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-24T12:01:55.606 INFO:teuthology.orchestra.run.smithi006.stdout: librados-devel x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 127 k 2024-03-24T12:01:55.606 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:55.606 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-24T12:01:55.606 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:01:55.606 INFO:teuthology.orchestra.run.smithi006.stdout:Install 1 Package 2024-03-24T12:01:55.606 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:55.606 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 127 k 2024-03-24T12:01:55.606 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 456 k 2024-03-24T12:01:55.606 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-24T12:01:55.842 INFO:teuthology.orchestra.run.smithi006.stdout:librados-devel-19.0.0-1667.gdb0330b1.el9.x86_64 538 kB/s | 127 kB 00:00 2024-03-24T12:01:55.843 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:01:55.843 INFO:teuthology.orchestra.run.smithi006.stdout:Total 535 kB/s | 127 kB 00:00 2024-03-24T12:01:55.843 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-24T12:01:55.848 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-24T12:01:55.849 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-24T12:01:55.907 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-24T12:01:55.907 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-24T12:01:56.098 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-24T12:01:56.201 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librados-devel-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:01:56.608 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librados-devel-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:01:56.890 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados-devel-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:01:56.890 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:56.890 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-24T12:01:56.890 INFO:teuthology.orchestra.run.smithi006.stdout: librados-devel-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:56.890 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:56.891 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:01:57.135 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install libcephfs2 2024-03-24T12:01:57.714 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:43 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:01:57.815 INFO:teuthology.orchestra.run.smithi006.stdout:Package libcephfs2-2:19.0.0-1667.gdb0330b1.el9.x86_64 is already installed. 2024-03-24T12:01:57.870 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:01:57.871 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2024-03-24T12:01:57.871 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:01:57.922 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install libcephfs-devel 2024-03-24T12:01:58.084 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1667.gdb033 12/12 2024-03-24T12:01:58.084 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1667.gdb033 1/12 2024-03-24T12:01:58.084 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-24T12:01:58.084 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-24T12:01:58.084 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-24T12:01:58.084 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-24T12:01:58.084 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-24T12:01:58.084 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-24T12:01:58.085 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-24T12:01:58.085 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-24T12:01:58.085 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-24T12:01:58.085 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-24T12:01:58.492 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:44 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:01:58.531 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-24T12:01:58.531 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:58.531 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-03-24T12:01:58.531 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1667.gdb0330b1.el9.noarch 2024-03-24T12:01:58.531 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-24T12:01:58.531 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-24T12:01:58.531 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-24T12:01:58.531 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-24T12:01:58.531 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-24T12:01:58.532 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-24T12:01:58.532 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-24T12:01:58.532 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-24T12:01:58.532 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-24T12:01:58.532 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-24T12:01:58.532 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-24T12:01:58.532 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:58.532 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:01:58.647 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:01:58.648 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:01:58.648 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repo Size 2024-03-24T12:01:58.648 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:01:58.648 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-24T12:01:58.649 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs-devel x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 31 k 2024-03-24T12:01:58.649 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:58.649 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-24T12:01:58.649 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:01:58.649 INFO:teuthology.orchestra.run.smithi006.stdout:Install 1 Package 2024-03-24T12:01:58.649 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:58.649 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 31 k 2024-03-24T12:01:58.649 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 140 k 2024-03-24T12:01:58.649 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-24T12:01:58.894 INFO:teuthology.orchestra.run.smithi006.stdout:libcephfs-devel-19.0.0-1667.gdb0330b1.el9.x86_6 127 kB/s | 31 kB 00:00 2024-03-24T12:01:58.895 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:01:58.895 INFO:teuthology.orchestra.run.smithi006.stdout:Total 126 kB/s | 31 kB 00:00 2024-03-24T12:01:58.895 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-24T12:01:58.899 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-24T12:01:58.899 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-24T12:01:58.933 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-24T12:01:58.933 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-24T12:01:59.033 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-24T12:01:59.056 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr-rook 2024-03-24T12:01:59.153 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephfs-devel-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:01:59.500 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:01:59.626 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:21 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:01:59.749 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephfs-devel-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:01:59.749 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:59.749 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-24T12:01:59.749 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs-devel-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:01:59.749 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:01:59.749 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:01:59.777 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:01:59.778 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================== 2024-03-24T12:01:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-03-24T12:01:59.778 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================== 2024-03-24T12:01:59.779 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-03-24T12:01:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook noarch 2:19.0.0-1667.gdb0330b1.el9 ceph-noarch 49 k 2024-03-24T12:01:59.779 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-03-24T12:01:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-24T12:01:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-24T12:01:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-24T12:01:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-24T12:01:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-24T12:01:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-24T12:01:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-24T12:01:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-24T12:01:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-24T12:01:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-24T12:01:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-24T12:01:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-24T12:01:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:59.780 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-03-24T12:01:59.780 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================== 2024-03-24T12:01:59.780 INFO:teuthology.orchestra.run.smithi090.stdout:Install 13 Packages 2024-03-24T12:01:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:01:59.781 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 2.0 M 2024-03-24T12:01:59.781 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 26 M 2024-03-24T12:01:59.781 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-03-24T12:01:59.933 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install librados2 2024-03-24T12:02:00.274 INFO:teuthology.orchestra.run.smithi090.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 171 kB/s | 24 kB 00:00 2024-03-24T12:02:00.300 INFO:teuthology.orchestra.run.smithi090.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 108 kB/s | 18 kB 00:00 2024-03-24T12:02:00.350 INFO:teuthology.orchestra.run.smithi090.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-03-24T12:02:00.375 INFO:teuthology.orchestra.run.smithi090.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-24T12:02:00.401 INFO:teuthology.orchestra.run.smithi090.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-03-24T12:02:00.426 INFO:teuthology.orchestra.run.smithi090.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 854 kB/s | 43 kB 00:00 2024-03-24T12:02:00.451 INFO:teuthology.orchestra.run.smithi090.stdout:(7/13): ceph-mgr-rook-19.0.0-1667.gdb0330b1.el9 154 kB/s | 49 kB 00:00 2024-03-24T12:02:00.505 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:46 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:02:00.585 INFO:teuthology.orchestra.run.smithi090.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 94 kB/s | 15 kB 00:00 2024-03-24T12:02:00.603 INFO:teuthology.orchestra.run.smithi006.stdout:Package librados2-2:19.0.0-1667.gdb0330b1.el9.x86_64 is already installed. 2024-03-24T12:02:00.610 INFO:teuthology.orchestra.run.smithi090.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 154 kB/s | 32 kB 00:00 2024-03-24T12:02:00.657 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:02:00.658 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2024-03-24T12:02:00.659 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:02:00.661 INFO:teuthology.orchestra.run.smithi090.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 964 kB/s | 203 kB 00:00 2024-03-24T12:02:00.687 INFO:teuthology.orchestra.run.smithi090.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 773 kB/s | 59 kB 00:00 2024-03-24T12:02:00.722 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install librbd1 2024-03-24T12:02:00.737 INFO:teuthology.orchestra.run.smithi090.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 6.7 MB/s | 1.0 MB 00:00 2024-03-24T12:02:00.762 INFO:teuthology.orchestra.run.smithi090.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 894 kB/s | 90 kB 00:00 2024-03-24T12:02:00.766 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:02:00.766 INFO:teuthology.orchestra.run.smithi090.stdout:Total 2.1 MB/s | 2.0 MB 00:00 2024-03-24T12:02:00.767 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-03-24T12:02:00.779 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-03-24T12:02:00.779 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-03-24T12:02:00.868 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-03-24T12:02:00.868 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-03-24T12:02:01.293 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:47 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:02:01.374 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-03-24T12:02:01.392 INFO:teuthology.orchestra.run.smithi006.stdout:Package librbd1-2:19.0.0-1667.gdb0330b1.el9.x86_64 is already installed. 2024-03-24T12:02:01.447 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:02:01.448 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2024-03-24T12:02:01.448 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:02:01.502 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install python3-rados 2024-03-24T12:02:01.542 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-24T12:02:01.597 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-24T12:02:01.661 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-24T12:02:01.699 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-24T12:02:01.740 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-24T12:02:01.835 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-24T12:02:01.942 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-24T12:02:02.002 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-24T12:02:02.076 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:48 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:02:02.175 INFO:teuthology.orchestra.run.smithi006.stdout:Package python3-rados-2:19.0.0-1667.gdb0330b1.el9.x86_64 is already installed. 2024-03-24T12:02:02.229 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:02:02.231 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2024-03-24T12:02:02.231 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:02:02.297 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install python3-rgw 2024-03-24T12:02:02.719 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-24T12:02:02.784 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-24T12:02:02.818 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-24T12:02:02.871 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:48 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:02:02.874 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-24T12:02:02.904 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-rook-2:19.0.0-1667.gdb0330b1.el9.noarch 13/13 2024-03-24T12:02:02.971 INFO:teuthology.orchestra.run.smithi006.stdout:Package python3-rgw-2:19.0.0-1667.gdb0330b1.el9.x86_64 is already installed. 2024-03-24T12:02:03.025 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:02:03.026 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2024-03-24T12:02:03.026 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:02:03.085 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install python3-cephfs 2024-03-24T12:02:03.656 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:49 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:02:03.757 INFO:teuthology.orchestra.run.smithi006.stdout:Package python3-cephfs-2:19.0.0-1667.gdb0330b1.el9.x86_64 is already installed. 2024-03-24T12:02:03.811 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:02:03.813 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2024-03-24T12:02:03.813 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:02:03.869 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install python3-rbd 2024-03-24T12:02:03.912 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1667.gdb0330b1.el9.noarch 13/13 2024-03-24T12:02:03.912 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1667.gdb0330b1.el9.noarch 1/13 2024-03-24T12:02:03.912 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-24T12:02:03.913 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-24T12:02:03.913 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-24T12:02:03.913 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-24T12:02:03.913 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-24T12:02:03.913 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-24T12:02:03.913 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-24T12:02:03.913 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-24T12:02:03.913 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-24T12:02:03.913 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-24T12:02:03.913 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-24T12:02:04.273 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-24T12:02:04.273 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:04.273 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-03-24T12:02:04.273 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook-2:19.0.0-1667.gdb0330b1.el9.noarch 2024-03-24T12:02:04.273 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-24T12:02:04.274 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-24T12:02:04.274 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-24T12:02:04.274 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-24T12:02:04.274 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-24T12:02:04.274 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-24T12:02:04.274 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-24T12:02:04.274 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-24T12:02:04.274 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-24T12:02:04.274 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-24T12:02:04.274 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-24T12:02:04.274 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-24T12:02:04.274 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:04.274 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:02:04.442 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:50 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:02:04.541 INFO:teuthology.orchestra.run.smithi006.stdout:Package python3-rbd-2:19.0.0-1667.gdb0330b1.el9.x86_64 is already installed. 2024-03-24T12:02:04.552 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr-cephadm 2024-03-24T12:02:04.596 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:02:04.597 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2024-03-24T12:02:04.597 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:02:04.655 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install rbd-fuse 2024-03-24T12:02:05.112 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:27 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:02:05.224 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:51 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:02:05.261 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:02:05.262 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================== 2024-03-24T12:02:05.262 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-03-24T12:02:05.262 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================== 2024-03-24T12:02:05.262 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-03-24T12:02:05.262 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1667.gdb0330b1.el9 ceph-noarch 138 k 2024-03-24T12:02:05.262 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-03-24T12:02:05.262 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-24T12:02:05.263 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-24T12:02:05.263 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-24T12:02:05.263 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-24T12:02:05.263 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-24T12:02:05.263 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:05.263 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-03-24T12:02:05.263 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================== 2024-03-24T12:02:05.263 INFO:teuthology.orchestra.run.smithi090.stdout:Install 6 Packages 2024-03-24T12:02:05.263 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:05.264 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 6.8 M 2024-03-24T12:02:05.264 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 33 M 2024-03-24T12:02:05.264 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-03-24T12:02:05.376 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:02:05.377 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:02:05.377 INFO:teuthology.orchestra.run.smithi006.stdout: Package Architecture Version Repository Size 2024-03-24T12:02:05.377 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:02:05.378 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-24T12:02:05.378 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-fuse x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 85 k 2024-03-24T12:02:05.378 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:02:05.378 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-24T12:02:05.378 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:02:05.378 INFO:teuthology.orchestra.run.smithi006.stdout:Install 1 Package 2024-03-24T12:02:05.378 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:02:05.378 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 85 k 2024-03-24T12:02:05.378 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 231 k 2024-03-24T12:02:05.378 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-24T12:02:05.697 INFO:teuthology.orchestra.run.smithi006.stdout:rbd-fuse-19.0.0-1667.gdb0330b1.el9.x86_64.rpm 267 kB/s | 85 kB 00:00 2024-03-24T12:02:05.698 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:02:05.698 INFO:teuthology.orchestra.run.smithi006.stdout:Total 266 kB/s | 85 kB 00:00 2024-03-24T12:02:05.698 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-24T12:02:05.706 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-24T12:02:05.706 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-24T12:02:05.763 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-24T12:02:05.764 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-24T12:02:05.796 INFO:teuthology.orchestra.run.smithi090.stdout:(1/6): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 5.8 MB 00:00 2024-03-24T12:02:05.904 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-24T12:02:05.922 INFO:teuthology.orchestra.run.smithi090.stdout:(2/6): ceph-mgr-cephadm-19.0.0-1667.gdb0330b1.e 333 kB/s | 138 kB 00:00 2024-03-24T12:02:05.990 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-fuse-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:06.072 INFO:teuthology.orchestra.run.smithi090.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 387 kB/s | 58 kB 00:00 2024-03-24T12:02:06.140 INFO:teuthology.orchestra.run.smithi090.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 751 kB/s | 51 kB 00:00 2024-03-24T12:02:06.207 INFO:teuthology.orchestra.run.smithi090.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.4 MB/s | 548 kB 00:00 2024-03-24T12:02:06.342 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: rbd-fuse-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:06.541 INFO:teuthology.orchestra.run.smithi090.stdout:(6/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 240 kB/s | 248 kB 00:01 2024-03-24T12:02:06.543 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:02:06.543 INFO:teuthology.orchestra.run.smithi090.stdout:Total 5.3 MB/s | 6.8 MB 00:01 2024-03-24T12:02:06.557 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-03-24T12:02:06.568 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-03-24T12:02:06.568 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-03-24T12:02:06.631 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-fuse-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:06.631 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:02:06.631 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-24T12:02:06.631 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-fuse-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:02:06.631 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:02:06.631 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:02:06.657 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-03-24T12:02:06.657 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-03-24T12:02:06.915 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install rbd-mirror 2024-03-24T12:02:06.955 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-03-24T12:02:07.130 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-24T12:02:07.176 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-24T12:02:07.507 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:53 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:02:07.661 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:02:07.662 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:02:07.662 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repo Size 2024-03-24T12:02:07.662 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:02:07.662 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-24T12:02:07.663 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-mirror x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 3.1 M 2024-03-24T12:02:07.663 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:02:07.663 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-24T12:02:07.663 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:02:07.663 INFO:teuthology.orchestra.run.smithi006.stdout:Install 1 Package 2024-03-24T12:02:07.663 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:02:07.663 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 3.1 M 2024-03-24T12:02:07.663 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 13 M 2024-03-24T12:02:07.663 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-24T12:02:07.698 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-24T12:02:07.795 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-24T12:02:07.865 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-24T12:02:07.897 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1667.gdb0330b1.el9.noarch 6/6 2024-03-24T12:02:08.068 INFO:teuthology.orchestra.run.smithi006.stdout:rbd-mirror-19.0.0-1667.gdb0330b1.el9.x86_64.rpm 7.6 MB/s | 3.1 MB 00:00 2024-03-24T12:02:08.068 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:02:08.068 INFO:teuthology.orchestra.run.smithi006.stdout:Total 7.6 MB/s | 3.1 MB 00:00 2024-03-24T12:02:08.069 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-24T12:02:08.078 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-24T12:02:08.078 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-24T12:02:08.154 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-24T12:02:08.154 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-24T12:02:08.595 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-24T12:02:08.652 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-mirror-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:08.677 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:08.677 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T12:02:08.677 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-24T12:02:08.677 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-24T12:02:08.677 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-24T12:02:08.677 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:02:09.093 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1667.gdb0330b1.el9.noarch 6/6 2024-03-24T12:02:09.094 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1667.gdb0330b1.el9.noarch 1/6 2024-03-24T12:02:09.094 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-24T12:02:09.094 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-24T12:02:09.094 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-24T12:02:09.094 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-24T12:02:09.391 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-24T12:02:09.391 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:09.391 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-03-24T12:02:09.391 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm-2:19.0.0-1667.gdb0330b1.el9.noarch 2024-03-24T12:02:09.392 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-24T12:02:09.392 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-24T12:02:09.392 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-24T12:02:09.392 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-24T12:02:09.392 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-24T12:02:09.392 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:09.392 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:02:09.666 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-fuse 2024-03-24T12:02:09.847 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-mirror-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:09.847 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:02:09.847 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-24T12:02:09.847 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-mirror-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:02:09.847 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:02:09.848 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:02:10.104 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install rbd-nbd 2024-03-24T12:02:10.223 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:32 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:02:10.372 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:02:10.373 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:02:10.373 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-03-24T12:02:10.373 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:02:10.373 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-03-24T12:02:10.373 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 886 k 2024-03-24T12:02:10.373 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-03-24T12:02:10.373 INFO:teuthology.orchestra.run.smithi090.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-24T12:02:10.373 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:10.373 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-03-24T12:02:10.373 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:02:10.373 INFO:teuthology.orchestra.run.smithi090.stdout:Install 2 Packages 2024-03-24T12:02:10.373 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:10.374 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 964 k 2024-03-24T12:02:10.374 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 2.8 M 2024-03-24T12:02:10.374 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-03-24T12:02:10.513 INFO:teuthology.orchestra.run.smithi090.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 568 kB/s | 79 kB 00:00 2024-03-24T12:02:10.680 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:56 ago on Sun 24 Mar 2024 12:00:14 PM UTC. 2024-03-24T12:02:10.833 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2024-03-24T12:02:10.834 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:02:10.834 INFO:teuthology.orchestra.run.smithi006.stdout: Package Architecture Version Repository Size 2024-03-24T12:02:10.834 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:02:10.834 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2024-03-24T12:02:10.834 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-nbd x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 171 k 2024-03-24T12:02:10.834 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:02:10.834 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2024-03-24T12:02:10.834 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2024-03-24T12:02:10.834 INFO:teuthology.orchestra.run.smithi006.stdout:Install 1 Package 2024-03-24T12:02:10.834 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:02:10.834 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 171 k 2024-03-24T12:02:10.835 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 494 k 2024-03-24T12:02:10.835 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2024-03-24T12:02:10.996 INFO:teuthology.orchestra.run.smithi090.stdout:(2/2): ceph-fuse-19.0.0-1667.gdb0330b1.el9.x86_ 1.4 MB/s | 886 kB 00:00 2024-03-24T12:02:10.997 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:02:10.998 INFO:teuthology.orchestra.run.smithi090.stdout:Total 1.5 MB/s | 964 kB 00:00 2024-03-24T12:02:10.998 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-03-24T12:02:11.020 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-03-24T12:02:11.020 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-03-24T12:02:11.054 INFO:teuthology.orchestra.run.smithi006.stdout:rbd-nbd-19.0.0-1667.gdb0330b1.el9.x86_64.rpm 779 kB/s | 171 kB 00:00 2024-03-24T12:02:11.055 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:02:11.055 INFO:teuthology.orchestra.run.smithi006.stdout:Total 773 kB/s | 171 kB 00:00 2024-03-24T12:02:11.055 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2024-03-24T12:02:11.063 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2024-03-24T12:02:11.064 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2024-03-24T12:02:11.095 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-03-24T12:02:11.095 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-03-24T12:02:11.122 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2024-03-24T12:02:11.122 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2024-03-24T12:02:11.270 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-03-24T12:02:11.272 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2024-03-24T12:02:11.381 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-nbd-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:11.414 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-24T12:02:11.512 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-fuse-2:19.0.0-1667.gdb0330b1.el9.x86_64 2/2 2024-03-24T12:02:11.813 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: rbd-nbd-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:12.098 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-nbd-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:12.098 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:02:12.098 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2024-03-24T12:02:12.098 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-nbd-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:02:12.098 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:02:12.098 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2024-03-24T12:02:12.294 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-fuse-2:19.0.0-1667.gdb0330b1.el9.x86_64 2/2 2024-03-24T12:02:12.294 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-fuse-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/2 2024-03-24T12:02:12.351 DEBUG:teuthology.parallel:result is None 2024-03-24T12:02:12.572 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-24T12:02:12.572 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:12.573 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-03-24T12:02:12.573 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse-2:19.0.0-1667.gdb0330b1.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-24T12:02:12.573 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:12.573 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:02:12.885 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-volume 2024-03-24T12:02:13.458 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:35 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:02:13.614 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:02:13.615 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:02:13.615 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-03-24T12:02:13.615 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:02:13.615 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-03-24T12:02:13.615 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume noarch 2:19.0.0-1667.gdb0330b1.el9 ceph-noarch 264 k 2024-03-24T12:02:13.615 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-03-24T12:02:13.616 INFO:teuthology.orchestra.run.smithi090.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-24T12:02:13.616 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:13.616 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-03-24T12:02:13.616 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:02:13.616 INFO:teuthology.orchestra.run.smithi090.stdout:Install 2 Packages 2024-03-24T12:02:13.616 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:13.616 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 332 k 2024-03-24T12:02:13.616 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 1.5 M 2024-03-24T12:02:13.616 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-03-24T12:02:13.740 INFO:teuthology.orchestra.run.smithi090.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 555 kB/s | 69 kB 00:00 2024-03-24T12:02:13.982 INFO:teuthology.orchestra.run.smithi090.stdout:(2/2): ceph-volume-19.0.0-1667.gdb0330b1.el9.no 721 kB/s | 264 kB 00:00 2024-03-24T12:02:13.983 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:02:13.983 INFO:teuthology.orchestra.run.smithi090.stdout:Total 904 kB/s | 332 kB 00:00 2024-03-24T12:02:13.983 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-03-24T12:02:13.992 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-03-24T12:02:13.992 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-03-24T12:02:14.032 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-03-24T12:02:14.033 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-03-24T12:02:14.180 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-03-24T12:02:14.355 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-03-24T12:02:14.385 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-volume-2:19.0.0-1667.gdb0330b1.el9.noarch 2/2 2024-03-24T12:02:14.400 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-volume-2:19.0.0-1667.gdb0330b1.el9.noarch 2/2 2024-03-24T12:02:14.400 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T12:02:14.400 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-24T12:02:14.401 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:15.237 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-volume-2:19.0.0-1667.gdb0330b1.el9.noarch 1/2 2024-03-24T12:02:15.553 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-03-24T12:02:15.553 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:15.554 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-03-24T12:02:15.554 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume-2:19.0.0-1667.gdb0330b1.el9.noarch 2024-03-24T12:02:15.554 INFO:teuthology.orchestra.run.smithi090.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-24T12:02:15.554 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:15.554 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:02:15.941 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install librados-devel 2024-03-24T12:02:16.512 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:38 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:02:16.664 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:02:16.665 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:02:16.665 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repo Size 2024-03-24T12:02:16.665 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:02:16.665 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-03-24T12:02:16.665 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 127 k 2024-03-24T12:02:16.665 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:16.665 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-03-24T12:02:16.665 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:02:16.665 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-03-24T12:02:16.665 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:16.665 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 127 k 2024-03-24T12:02:16.665 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 456 k 2024-03-24T12:02:16.665 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-03-24T12:02:16.869 INFO:teuthology.orchestra.run.smithi090.stdout:librados-devel-19.0.0-1667.gdb0330b1.el9.x86_64 623 kB/s | 127 kB 00:00 2024-03-24T12:02:16.869 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:02:16.870 INFO:teuthology.orchestra.run.smithi090.stdout:Total 619 kB/s | 127 kB 00:00 2024-03-24T12:02:16.870 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-03-24T12:02:16.875 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-03-24T12:02:16.875 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-03-24T12:02:16.930 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-03-24T12:02:16.930 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-03-24T12:02:17.146 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-03-24T12:02:17.251 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librados-devel-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:17.777 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados-devel-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:18.044 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados-devel-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:18.045 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:18.045 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-03-24T12:02:18.045 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:02:18.045 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:18.045 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:02:18.345 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install libcephfs2 2024-03-24T12:02:18.934 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:40 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:02:19.033 INFO:teuthology.orchestra.run.smithi090.stdout:Package libcephfs2-2:19.0.0-1667.gdb0330b1.el9.x86_64 is already installed. 2024-03-24T12:02:19.087 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:02:19.088 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-03-24T12:02:19.088 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:02:19.163 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install libcephfs-devel 2024-03-24T12:02:19.720 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:41 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:02:19.869 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:02:19.870 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:02:19.870 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repo Size 2024-03-24T12:02:19.870 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:02:19.870 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-03-24T12:02:19.870 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 31 k 2024-03-24T12:02:19.870 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:19.870 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-03-24T12:02:19.870 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:02:19.870 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-03-24T12:02:19.870 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:19.871 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 31 k 2024-03-24T12:02:19.871 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 140 k 2024-03-24T12:02:19.871 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-03-24T12:02:20.037 INFO:teuthology.orchestra.run.smithi090.stdout:libcephfs-devel-19.0.0-1667.gdb0330b1.el9.x86_6 188 kB/s | 31 kB 00:00 2024-03-24T12:02:20.037 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:02:20.037 INFO:teuthology.orchestra.run.smithi090.stdout:Total 186 kB/s | 31 kB 00:00 2024-03-24T12:02:20.038 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-03-24T12:02:20.041 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-03-24T12:02:20.041 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-03-24T12:02:20.073 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-03-24T12:02:20.073 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-03-24T12:02:20.174 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-03-24T12:02:20.276 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs-devel-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:20.773 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:21.028 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs-devel-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:21.028 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:21.028 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-03-24T12:02:21.028 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:02:21.028 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:21.029 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:02:21.251 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install librados2 2024-03-24T12:02:21.813 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:43 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:02:21.908 INFO:teuthology.orchestra.run.smithi090.stdout:Package librados2-2:19.0.0-1667.gdb0330b1.el9.x86_64 is already installed. 2024-03-24T12:02:21.958 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:02:21.959 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-03-24T12:02:21.959 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:02:22.063 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install librbd1 2024-03-24T12:02:22.620 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:44 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:02:22.716 INFO:teuthology.orchestra.run.smithi090.stdout:Package librbd1-2:19.0.0-1667.gdb0330b1.el9.x86_64 is already installed. 2024-03-24T12:02:22.767 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:02:22.768 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-03-24T12:02:22.768 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:02:22.884 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install python3-rados 2024-03-24T12:02:23.445 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:45 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:02:23.542 INFO:teuthology.orchestra.run.smithi090.stdout:Package python3-rados-2:19.0.0-1667.gdb0330b1.el9.x86_64 is already installed. 2024-03-24T12:02:23.596 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:02:23.597 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-03-24T12:02:23.597 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:02:23.670 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install python3-rgw 2024-03-24T12:02:24.234 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:46 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:02:24.330 INFO:teuthology.orchestra.run.smithi090.stdout:Package python3-rgw-2:19.0.0-1667.gdb0330b1.el9.x86_64 is already installed. 2024-03-24T12:02:24.382 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:02:24.383 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-03-24T12:02:24.383 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:02:24.473 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install python3-cephfs 2024-03-24T12:02:25.035 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:47 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:02:25.132 INFO:teuthology.orchestra.run.smithi090.stdout:Package python3-cephfs-2:19.0.0-1667.gdb0330b1.el9.x86_64 is already installed. 2024-03-24T12:02:25.184 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:02:25.185 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-03-24T12:02:25.185 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:02:25.289 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install python3-rbd 2024-03-24T12:02:25.850 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:47 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:02:25.946 INFO:teuthology.orchestra.run.smithi090.stdout:Package python3-rbd-2:19.0.0-1667.gdb0330b1.el9.x86_64 is already installed. 2024-03-24T12:02:25.998 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:02:25.999 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-03-24T12:02:25.999 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:02:26.082 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install rbd-fuse 2024-03-24T12:02:26.643 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:48 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:02:26.791 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:02:26.792 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:02:26.792 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-03-24T12:02:26.792 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:02:26.792 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-03-24T12:02:26.792 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 85 k 2024-03-24T12:02:26.792 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:26.793 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-03-24T12:02:26.793 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:02:26.793 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-03-24T12:02:26.793 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:26.793 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 85 k 2024-03-24T12:02:26.793 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 231 k 2024-03-24T12:02:26.793 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-03-24T12:02:27.076 INFO:teuthology.orchestra.run.smithi090.stdout:rbd-fuse-19.0.0-1667.gdb0330b1.el9.x86_64.rpm 301 kB/s | 85 kB 00:00 2024-03-24T12:02:27.076 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:02:27.077 INFO:teuthology.orchestra.run.smithi090.stdout:Total 300 kB/s | 85 kB 00:00 2024-03-24T12:02:27.077 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-03-24T12:02:27.084 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-03-24T12:02:27.084 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-03-24T12:02:27.139 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-03-24T12:02:27.139 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-03-24T12:02:27.276 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-03-24T12:02:27.382 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-fuse-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:27.843 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-fuse-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:28.126 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-fuse-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:28.127 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:28.127 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-03-24T12:02:28.127 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:02:28.127 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:28.127 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:02:28.422 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install rbd-mirror 2024-03-24T12:02:28.999 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:50 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:02:29.154 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:02:29.155 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:02:29.155 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repo Size 2024-03-24T12:02:29.155 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:02:29.155 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-03-24T12:02:29.155 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 3.1 M 2024-03-24T12:02:29.155 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:29.155 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-03-24T12:02:29.155 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:02:29.156 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-03-24T12:02:29.156 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:29.156 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 3.1 M 2024-03-24T12:02:29.156 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 13 M 2024-03-24T12:02:29.156 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-03-24T12:02:29.558 INFO:teuthology.orchestra.run.smithi090.stdout:rbd-mirror-19.0.0-1667.gdb0330b1.el9.x86_64.rpm 7.7 MB/s | 3.1 MB 00:00 2024-03-24T12:02:29.559 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:02:29.559 INFO:teuthology.orchestra.run.smithi090.stdout:Total 7.6 MB/s | 3.1 MB 00:00 2024-03-24T12:02:29.559 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-03-24T12:02:29.569 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-03-24T12:02:29.569 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-03-24T12:02:29.641 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-03-24T12:02:29.642 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-03-24T12:02:30.059 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-03-24T12:02:30.136 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-mirror-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:30.160 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:30.160 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-24T12:02:30.160 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-24T12:02:30.160 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-24T12:02:30.160 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-24T12:02:30.160 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:31.420 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-mirror-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:31.420 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:31.420 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-03-24T12:02:31.420 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:02:31.420 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:31.420 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:02:31.739 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install rbd-nbd 2024-03-24T12:02:32.317 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:54 ago on Sun 24 Mar 2024 12:00:38 PM UTC. 2024-03-24T12:02:32.463 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-24T12:02:32.464 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:02:32.464 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-03-24T12:02:32.464 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:02:32.464 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-03-24T12:02:32.464 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd x86_64 2:19.0.0-1667.gdb0330b1.el9 ceph 171 k 2024-03-24T12:02:32.464 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:32.464 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-03-24T12:02:32.464 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-24T12:02:32.464 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-03-24T12:02:32.464 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:32.465 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 171 k 2024-03-24T12:02:32.465 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 494 k 2024-03-24T12:02:32.465 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-03-24T12:02:32.668 INFO:teuthology.orchestra.run.smithi090.stdout:rbd-nbd-19.0.0-1667.gdb0330b1.el9.x86_64.rpm 843 kB/s | 171 kB 00:00 2024-03-24T12:02:32.668 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:02:32.668 INFO:teuthology.orchestra.run.smithi090.stdout:Total 837 kB/s | 171 kB 00:00 2024-03-24T12:02:32.668 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-03-24T12:02:32.676 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-03-24T12:02:32.676 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-03-24T12:02:32.732 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-03-24T12:02:32.733 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-03-24T12:02:32.878 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-03-24T12:02:32.974 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-nbd-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:33.506 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-nbd-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:33.818 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-nbd-2:19.0.0-1667.gdb0330b1.el9.x86_64 1/1 2024-03-24T12:02:33.818 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:33.818 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-03-24T12:02:33.818 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd-2:19.0.0-1667.gdb0330b1.el9.x86_64 2024-03-24T12:02:33.818 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:02:33.818 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-24T12:02:34.148 DEBUG:teuthology.parallel:result is None 2024-03-24T12:02:34.149 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T12:02:34.294 DEBUG:teuthology.orchestra.run.smithi006:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-24T12:02:34.318 INFO:teuthology.orchestra.run.smithi006.stdout:19.0.0-1667.gdb0330b1.el9 2024-03-24T12:02:34.319 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1667.gdb0330b1.el9 2024-03-24T12:02:34.319 INFO:teuthology.task.install:The correct ceph version 19.0.0-1667.gdb0330b1 is installed. 2024-03-24T12:02:34.321 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T12:02:34.462 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-24T12:02:34.482 INFO:teuthology.orchestra.run.smithi090.stdout:19.0.0-1667.gdb0330b1.el9 2024-03-24T12:02:34.483 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1667.gdb0330b1.el9 2024-03-24T12:02:34.483 INFO:teuthology.task.install:The correct ceph version 19.0.0-1667.gdb0330b1 is installed. 2024-03-24T12:02:34.484 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-24T12:02:34.485 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-24T12:02:34.485 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-24T12:02:34.517 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-24T12:02:34.517 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-24T12:02:34.549 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-24T12:02:34.550 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-24T12:02:34.550 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/daemon-helper 2024-03-24T12:02:34.587 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-24T12:02:34.656 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-24T12:02:34.657 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/daemon-helper 2024-03-24T12:02:34.683 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-24T12:02:34.750 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-24T12:02:34.750 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-24T12:02:34.750 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-24T12:02:34.779 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-24T12:02:34.848 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-24T12:02:34.848 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-24T12:02:34.875 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-24T12:02:34.942 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-24T12:02:34.942 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-24T12:02:34.942 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/stdin-killer 2024-03-24T12:02:34.970 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-24T12:02:35.038 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-24T12:02:35.039 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/stdin-killer 2024-03-24T12:02:35.065 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-24T12:02:35.132 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-24T12:02:35.228 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 3, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': 'db0330b1e4e2470d52b750e251e55a522b4f7d69', 'cephadm_mode': 'root'} 2024-03-24T12:02:35.228 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T12:02:35.229 INFO:tasks.cephadm:Cluster fsid is 66d0ad74-e9d6-11ee-95cd-87774f69a715 2024-03-24T12:02:35.229 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-24T12:02:35.229 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.6', 'mon.c': '[v2:172.21.15.6:3301,v1:172.21.15.6:6790]', 'mon.b': '172.21.15.90'} 2024-03-24T12:02:35.229 INFO:tasks.cephadm:First mon is mon.a on smithi006 2024-03-24T12:02:35.229 INFO:tasks.cephadm:First mgr is y 2024-03-24T12:02:35.229 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-24T12:02:35.229 DEBUG:teuthology.orchestra.run.smithi006:> sudo hostname $(hostname -s) 2024-03-24T12:02:35.259 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname $(hostname -s) 2024-03-24T12:02:35.287 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-24T12:02:35.287 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T12:02:35.504 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'db0330b1e4e2470d52b750e251e55a522b4f7d69', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77704/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1667-gdb0330b1', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1667.gdb0330b1'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/db0330b1e4e2470d52b750e251e55a522b4f7d69/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-16 11:55:36.797747', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/db0330b1e4e2470d52b750e251e55a522b4f7d69/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-03-24T12:02:35.644 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 db0330b1e4e2470d52b750e251e55a522b4f7d69 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T12:02:35.645 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/db0330b1e4e2470d52b750e251e55a522b4f7d69/centos/9/x86_64/flavors/default/cephadm 2024-03-24T12:02:35.646 DEBUG:teuthology.orchestra.run.smithi006:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/db0330b1e4e2470d52b750e251e55a522b4f7d69/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-24T12:02:36.492 INFO:teuthology.orchestra.run.smithi006.stdout:-rw-r--r--. 1 ubuntu ubuntu 766879 Mar 24 12:02 /home/ubuntu/cephtest/cephadm 2024-03-24T12:02:36.492 DEBUG:teuthology.orchestra.run.smithi090:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/db0330b1e4e2470d52b750e251e55a522b4f7d69/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-24T12:02:37.409 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-r--r--. 1 ubuntu ubuntu 766879 Mar 24 12:02 /home/ubuntu/cephtest/cephadm 2024-03-24T12:02:37.409 DEBUG:teuthology.orchestra.run.smithi006:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-24T12:02:37.427 DEBUG:teuthology.orchestra.run.smithi090:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-24T12:02:37.458 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 on all hosts... 2024-03-24T12:02:37.458 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 pull 2024-03-24T12:02:37.469 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 pull 2024-03-24T12:02:38.271 INFO:teuthology.orchestra.run.smithi006.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69... 2024-03-24T12:02:38.494 INFO:teuthology.orchestra.run.smithi090.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69... 2024-03-24T12:03:33.021 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2024-03-24T12:03:33.021 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_version": "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)", 2024-03-24T12:03:33.021 INFO:teuthology.orchestra.run.smithi006.stdout: "image_id": "4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d", 2024-03-24T12:03:33.021 INFO:teuthology.orchestra.run.smithi006.stdout: "repo_digests": [ 2024-03-24T12:03:33.021 INFO:teuthology.orchestra.run.smithi006.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10" 2024-03-24T12:03:33.021 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2024-03-24T12:03:33.022 INFO:teuthology.orchestra.run.smithi006.stdout:} 2024-03-24T12:03:33.299 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-03-24T12:03:33.299 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_version": "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)", 2024-03-24T12:03:33.299 INFO:teuthology.orchestra.run.smithi090.stdout: "image_id": "4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d", 2024-03-24T12:03:33.299 INFO:teuthology.orchestra.run.smithi090.stdout: "repo_digests": [ 2024-03-24T12:03:33.299 INFO:teuthology.orchestra.run.smithi090.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10" 2024-03-24T12:03:33.299 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-03-24T12:03:33.299 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-03-24T12:03:33.323 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /etc/ceph 2024-03-24T12:03:33.353 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /etc/ceph 2024-03-24T12:03:33.383 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 777 /etc/ceph 2024-03-24T12:03:33.422 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 777 /etc/ceph 2024-03-24T12:03:33.449 INFO:tasks.cephadm:Writing seed config... 2024-03-24T12:03:33.450 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-24T12:03:33.451 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-24T12:03:33.451 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-24T12:03:33.451 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-03-24T12:03:33.451 INFO:tasks.cephadm: override: [global] ms type = async 2024-03-24T12:03:33.451 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-24T12:03:33.451 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-24T12:03:33.451 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-24T12:03:33.451 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-03-24T12:03:33.451 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-24T12:03:33.451 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-24T12:03:33.451 INFO:tasks.cephadm: override: [osd] osd class default list = * 2024-03-24T12:03:33.451 INFO:tasks.cephadm: override: [osd] osd class load list = * 2024-03-24T12:03:33.451 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-24T12:03:33.452 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-24T12:03:33.452 DEBUG:teuthology.orchestra.run.smithi006:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-24T12:03:33.477 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 66d0ad74-e9d6-11ee-95cd-87774f69a715 mon election default strategy = 3 ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd class default list = * osd class load list = * [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-03-24T12:03:33.477 DEBUG:teuthology.orchestra.run.smithi006:mon.a> sudo journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mon.a.service 2024-03-24T12:03:33.520 DEBUG:teuthology.orchestra.run.smithi006:mgr.y> sudo journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mgr.y.service 2024-03-24T12:03:33.562 INFO:tasks.cephadm:Bootstrapping... 2024-03-24T12:03:33.562 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 -v bootstrap --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.6 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-24T12:03:33.759 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2024-03-24T12:03:33.759 INFO:teuthology.orchestra.run.smithi006.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69', '-v', 'bootstrap', '--fsid', '66d0ad74-e9d6-11ee-95cd-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.6', '--skip-admin-label'] 2024-03-24T12:03:33.783 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-24T12:03:33.783 INFO:teuthology.orchestra.run.smithi006.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-24T12:03:33.783 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying podman|docker is present... 2024-03-24T12:03:33.806 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-24T12:03:33.807 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying lvm2 is present... 2024-03-24T12:03:33.807 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying time synchronization is in place... 2024-03-24T12:03:33.814 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-24T12:03:33.814 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-24T12:03:33.820 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-24T12:03:33.820 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-03-24T12:03:33.826 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout enabled 2024-03-24T12:03:33.832 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout active 2024-03-24T12:03:33.832 INFO:teuthology.orchestra.run.smithi006.stdout:Unit chronyd.service is enabled and running 2024-03-24T12:03:33.832 INFO:teuthology.orchestra.run.smithi006.stdout:Repeating the final host check... 2024-03-24T12:03:33.854 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-24T12:03:33.855 INFO:teuthology.orchestra.run.smithi006.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-24T12:03:33.855 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl is present 2024-03-24T12:03:33.855 INFO:teuthology.orchestra.run.smithi006.stdout:lvcreate is present 2024-03-24T12:03:33.861 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-24T12:03:33.861 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-24T12:03:33.867 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-24T12:03:33.867 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-03-24T12:03:33.873 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout enabled 2024-03-24T12:03:33.879 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout active 2024-03-24T12:03:33.879 INFO:teuthology.orchestra.run.smithi006.stdout:Unit chronyd.service is enabled and running 2024-03-24T12:03:33.879 INFO:teuthology.orchestra.run.smithi006.stdout:Host looks OK 2024-03-24T12:03:33.879 INFO:teuthology.orchestra.run.smithi006.stdout:Cluster fsid: 66d0ad74-e9d6-11ee-95cd-87774f69a715 2024-03-24T12:03:33.879 INFO:teuthology.orchestra.run.smithi006.stdout:Acquiring lock 139931600961488 on /run/cephadm/66d0ad74-e9d6-11ee-95cd-87774f69a715.lock 2024-03-24T12:03:33.879 INFO:teuthology.orchestra.run.smithi006.stdout:Lock 139931600961488 acquired on /run/cephadm/66d0ad74-e9d6-11ee-95cd-87774f69a715.lock 2024-03-24T12:03:33.880 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying IP 172.21.15.6 port 3300 ... 2024-03-24T12:03:33.880 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying IP 172.21.15.6 port 6789 ... 2024-03-24T12:03:33.880 INFO:teuthology.orchestra.run.smithi006.stdout:Base mon IP(s) is [172.21.15.6:3300, 172.21.15.6:6789], mon addrv is [v2:172.21.15.6:3300,v1:172.21.15.6:6789] 2024-03-24T12:03:33.883 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.6 metric 100 2024-03-24T12:03:33.883 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.6 metric 100 2024-03-24T12:03:33.885 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-24T12:03:33.885 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-24T12:03:33.885 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-24T12:03:33.888 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-24T12:03:33.888 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-24T12:03:33.888 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-24T12:03:33.888 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-24T12:03:33.888 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15e4/64 scope link noprefixroute 2024-03-24T12:03:33.888 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-24T12:03:33.889 INFO:teuthology.orchestra.run.smithi006.stdout:Mon IP `172.21.15.6` is in CIDR network `172.21.0.0/20` 2024-03-24T12:03:33.889 INFO:teuthology.orchestra.run.smithi006.stdout:Mon IP `172.21.15.6` is in CIDR network `172.21.0.0/20` 2024-03-24T12:03:33.889 INFO:teuthology.orchestra.run.smithi006.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-24T12:03:33.889 INFO:teuthology.orchestra.run.smithi006.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-24T12:03:33.890 INFO:teuthology.orchestra.run.smithi006.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69... 2024-03-24T12:03:34.366 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stdout 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d 2024-03-24T12:03:34.366 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69... 2024-03-24T12:03:34.367 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Getting image source signatures 2024-03-24T12:03:34.367 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Copying blob sha256:989de1c398e87dbcb4906791a352373dc0e0415d619bc5b36f1f1823df929cb1 2024-03-24T12:03:34.367 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Copying blob sha256:f58b4670ef4403d11d1c912aa26a4dbcf73447df5302e9dc66f6a7fc0d9d1e47 2024-03-24T12:03:34.367 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Copying config sha256:4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d 2024-03-24T12:03:34.367 INFO:teuthology.orchestra.run.smithi006.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-24T12:03:35.311 INFO:teuthology.orchestra.run.smithi006.stdout:ceph: stdout ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev) 2024-03-24T12:03:35.312 INFO:teuthology.orchestra.run.smithi006.stdout:Ceph version: ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev) 2024-03-24T12:03:35.312 INFO:teuthology.orchestra.run.smithi006.stdout:Extracting ceph user uid/gid from container image... 2024-03-24T12:03:36.200 INFO:teuthology.orchestra.run.smithi006.stdout:stat: stdout 167 167 2024-03-24T12:03:36.200 INFO:teuthology.orchestra.run.smithi006.stdout:Creating initial keys... 2024-03-24T12:03:37.206 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph-authtool: stdout AQCYFgBmTfsWIRAAysxqBu2cTrIx7Hv5Vrhcqg== 2024-03-24T12:03:38.195 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph-authtool: stdout AQCZFgBmLCpzHxAAIVD3LsKBrACTOfbK1hx7wg== 2024-03-24T12:03:39.034 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph-authtool: stdout AQCaFgBmIaw/IBAApJlYb2UN8fFIJeO/VfLoGA== 2024-03-24T12:03:39.034 INFO:teuthology.orchestra.run.smithi006.stdout:Creating initial monmap... 2024-03-24T12:03:40.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-24T12:03:40.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-24T12:03:40.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 66d0ad74-e9d6-11ee-95cd-87774f69a715 2024-03-24T12:03:40.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-24T12:03:40.007 INFO:teuthology.orchestra.run.smithi006.stdout:monmaptool for a [v2:172.21.15.6:3300,v1:172.21.15.6:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-24T12:03:40.007 INFO:teuthology.orchestra.run.smithi006.stdout:setting min_mon_release = quincy 2024-03-24T12:03:40.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: set fsid to 66d0ad74-e9d6-11ee-95cd-87774f69a715 2024-03-24T12:03:40.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-24T12:03:40.008 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:03:40.008 INFO:teuthology.orchestra.run.smithi006.stdout:Creating mon... 2024-03-24T12:03:40.903 INFO:teuthology.orchestra.run.smithi006.stdout:create mon.a on 2024-03-24T12:03:41.102 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-24T12:03:41.243 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-24T12:03:41.415 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715.target. 2024-03-24T12:03:41.415 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715.target. 2024-03-24T12:03:41.639 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mon.a 2024-03-24T12:03:41.640 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to reset failed state of unit ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mon.a.service: Unit ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mon.a.service not loaded. 2024-03-24T12:03:41.793 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715.target.wants/ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mon.a.service → /etc/systemd/system/ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@.service. 2024-03-24T12:03:42.388 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-24T12:03:42.388 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2024-03-24T12:03:42.395 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-24T12:03:42.395 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-03-24T12:03:42.395 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2024-03-24T12:03:42.395 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to enable service . firewalld.service is not available 2024-03-24T12:03:42.396 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mon to start... 2024-03-24T12:03:42.396 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mon... 2024-03-24T12:03:43.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:43 smithi006 ceph-mon[29616]: from='client.? 172.21.15.6:0/2984204055' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-24T12:03:46.996 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout cluster: 2024-03-24T12:03:47.003 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout id: 66d0ad74-e9d6-11ee-95cd-87774f69a715 2024-03-24T12:03:47.003 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-24T12:03:47.003 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-24T12:03:47.003 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout services: 2024-03-24T12:03:47.003 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.600834s) 2024-03-24T12:03:47.003 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-24T12:03:47.003 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-24T12:03:47.003 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-24T12:03:47.004 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout data: 2024-03-24T12:03:47.004 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-24T12:03:47.004 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-24T12:03:47.004 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-24T12:03:47.004 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout pgs: 2024-03-24T12:03:47.004 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-24T12:03:47.004 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.856+0000 7fc30c18e700 1 Processor -- start 2024-03-24T12:03:47.004 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.856+0000 7fc30c18e700 1 -- start start 2024-03-24T12:03:47.004 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.856+0000 7fc30c18e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc304107130 0x7fc304107550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:03:47.004 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.856+0000 7fc30c18e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc304107b20 con 0x7fc304107130 2024-03-24T12:03:47.004 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.856+0000 7fc309f2a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc304107130 0x7fc304107550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:03:47.005 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.856+0000 7fc309f2a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc304107130 0x7fc304107550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40724/0 (socket says 172.21.15.6:40724) 2024-03-24T12:03:47.005 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.856+0000 7fc309f2a700 1 -- 172.21.15.6:0/2508466897 learned_addr learned my addr 172.21.15.6:0/2508466897 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:03:47.005 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.857+0000 7fc309f2a700 1 -- 172.21.15.6:0/2508466897 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc304105670 con 0x7fc304107130 2024-03-24T12:03:47.005 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.857+0000 7fc309f2a700 1 --2- 172.21.15.6:0/2508466897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc304107130 0x7fc304107550 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc3000098a0 tx=0x7fc300009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8664478829d34da server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:03:47.005 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.857+0000 7fc308f28700 1 -- 172.21.15.6:0/2508466897 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc300004160 con 0x7fc304107130 2024-03-24T12:03:47.005 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.857+0000 7fc308f28700 1 -- 172.21.15.6:0/2508466897 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3000042c0 con 0x7fc304107130 2024-03-24T12:03:47.005 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.857+0000 7fc308f28700 1 -- 172.21.15.6:0/2508466897 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3000045a0 con 0x7fc304107130 2024-03-24T12:03:47.005 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.858+0000 7fc30c18e700 1 -- 172.21.15.6:0/2508466897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc304107130 msgr2=0x7fc304107550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:03:47.005 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.858+0000 7fc30c18e700 1 --2- 172.21.15.6:0/2508466897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc304107130 0x7fc304107550 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc3000098a0 tx=0x7fc300009bb0 comp rx=0 tx=0).stop 2024-03-24T12:03:47.005 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.858+0000 7fc30c18e700 1 -- 172.21.15.6:0/2508466897 shutdown_connections 2024-03-24T12:03:47.005 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.858+0000 7fc30c18e700 1 --2- 172.21.15.6:0/2508466897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc304107130 0x7fc304107550 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:03:47.006 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.858+0000 7fc30c18e700 1 -- 172.21.15.6:0/2508466897 >> 172.21.15.6:0/2508466897 conn(0x7fc304101030 msgr2=0x7fc304103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:03:47.006 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.858+0000 7fc30c18e700 1 -- 172.21.15.6:0/2508466897 shutdown_connections 2024-03-24T12:03:47.006 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.858+0000 7fc30c18e700 1 -- 172.21.15.6:0/2508466897 wait complete. 2024-03-24T12:03:47.006 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.858+0000 7fc30c18e700 1 Processor -- start 2024-03-24T12:03:47.006 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.858+0000 7fc30c18e700 1 -- start start 2024-03-24T12:03:47.006 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.858+0000 7fc30c18e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc304107130 0x7fc304198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:03:47.006 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.858+0000 7fc30c18e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3041994e0 con 0x7fc304107130 2024-03-24T12:03:47.006 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.859+0000 7fc309f2a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc304107130 0x7fc304198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:03:47.006 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.859+0000 7fc309f2a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc304107130 0x7fc304198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40740/0 (socket says 172.21.15.6:40740) 2024-03-24T12:03:47.006 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.859+0000 7fc309f2a700 1 -- 172.21.15.6:0/2984204055 learned_addr learned my addr 172.21.15.6:0/2984204055 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:03:47.006 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.859+0000 7fc309f2a700 1 -- 172.21.15.6:0/2984204055 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc300009550 con 0x7fc304107130 2024-03-24T12:03:47.006 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.859+0000 7fc309f2a700 1 --2- 172.21.15.6:0/2984204055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc304107130 0x7fc304198fa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc300004000 tx=0x7fc300004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:03:47.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.859+0000 7fc2faffd700 1 -- 172.21.15.6:0/2984204055 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc300011630 con 0x7fc304107130 2024-03-24T12:03:47.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.859+0000 7fc2faffd700 1 -- 172.21.15.6:0/2984204055 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc300011790 con 0x7fc304107130 2024-03-24T12:03:47.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.859+0000 7fc2faffd700 1 -- 172.21.15.6:0/2984204055 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc300011a30 con 0x7fc304107130 2024-03-24T12:03:47.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.859+0000 7fc30c18e700 1 -- 172.21.15.6:0/2984204055 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3041996e0 con 0x7fc304107130 2024-03-24T12:03:47.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.859+0000 7fc30c18e700 1 -- 172.21.15.6:0/2984204055 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc304199b80 con 0x7fc304107130 2024-03-24T12:03:47.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.859+0000 7fc2faffd700 1 -- 172.21.15.6:0/2984204055 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc300022070 con 0x7fc304107130 2024-03-24T12:03:47.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.859+0000 7fc2faffd700 1 -- 172.21.15.6:0/2984204055 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc30001c450 con 0x7fc304107130 2024-03-24T12:03:47.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.860+0000 7fc30c18e700 1 -- 172.21.15.6:0/2984204055 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc304192c70 con 0x7fc304107130 2024-03-24T12:03:47.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.862+0000 7fc2faffd700 1 -- 172.21.15.6:0/2984204055 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc300011b90 con 0x7fc304107130 2024-03-24T12:03:47.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.936+0000 7fc30c18e700 1 -- 172.21.15.6:0/2984204055 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fc304105c10 con 0x7fc304107130 2024-03-24T12:03:47.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.936+0000 7fc2faffd700 1 -- 172.21.15.6:0/2984204055 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fc300011d30 con 0x7fc304107130 2024-03-24T12:03:47.007 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.937+0000 7fc30c18e700 1 -- 172.21.15.6:0/2984204055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc304107130 msgr2=0x7fc304198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:03:47.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.937+0000 7fc30c18e700 1 --2- 172.21.15.6:0/2984204055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc304107130 0x7fc304198fa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc300004000 tx=0x7fc300004750 comp rx=0 tx=0).stop 2024-03-24T12:03:47.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.937+0000 7fc30c18e700 1 -- 172.21.15.6:0/2984204055 shutdown_connections 2024-03-24T12:03:47.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.937+0000 7fc30c18e700 1 --2- 172.21.15.6:0/2984204055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc304107130 0x7fc304198fa0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:03:47.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.937+0000 7fc30c18e700 1 -- 172.21.15.6:0/2984204055 >> 172.21.15.6:0/2984204055 conn(0x7fc304101030 msgr2=0x7fc30418fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:03:47.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.937+0000 7fc30c18e700 1 -- 172.21.15.6:0/2984204055 shutdown_connections 2024-03-24T12:03:47.008 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:42.937+0000 7fc30c18e700 1 -- 172.21.15.6:0/2984204055 wait complete. 2024-03-24T12:03:47.008 INFO:teuthology.orchestra.run.smithi006.stdout:mon is available 2024-03-24T12:03:47.008 INFO:teuthology.orchestra.run.smithi006.stdout:Assimilating anything we can from ceph.conf... 2024-03-24T12:03:50.181 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:50 smithi006 ceph-mon[29616]: from='client.? 172.21.15.6:0/3606456592' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-24T12:03:50.182 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:50 smithi006 ceph-mon[29616]: from='client.? 172.21.15.6:0/3606456592' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-24T12:03:53.763 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-24T12:03:53.763 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [global] 2024-03-24T12:03:53.763 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout fsid = 66d0ad74-e9d6-11ee-95cd-87774f69a715 2024-03-24T12:03:53.763 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.6:3300,v1:172.21.15.6:6789] 2024-03-24T12:03:53.763 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-24T12:03:53.763 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-24T12:03:53.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-24T12:03:53.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-24T12:03:53.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-24T12:03:53.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-24T12:03:53.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-24T12:03:53.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-24T12:03:53.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [osd] 2024-03-24T12:03:53.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-24T12:03:53.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-24T12:03:53.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.029+0000 7fc2ff2d6700 1 Processor -- start 2024-03-24T12:03:53.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.029+0000 7fc2ff2d6700 1 -- start start 2024-03-24T12:03:53.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.029+0000 7fc2ff2d6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2f81064a0 0x7fc2f8108890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:03:53.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.029+0000 7fc2ff2d6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f8079720 con 0x7fc2f81064a0 2024-03-24T12:03:53.764 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.030+0000 7fc2fd072700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2f81064a0 0x7fc2f8108890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:03:53.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.030+0000 7fc2fd072700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2f81064a0 0x7fc2f8108890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57036/0 (socket says 172.21.15.6:57036) 2024-03-24T12:03:53.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.030+0000 7fc2fd072700 1 -- 172.21.15.6:0/3183945289 learned_addr learned my addr 172.21.15.6:0/3183945289 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:03:53.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.030+0000 7fc2fd072700 1 -- 172.21.15.6:0/3183945289 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f8108dd0 con 0x7fc2f81064a0 2024-03-24T12:03:53.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.030+0000 7fc2fd072700 1 --2- 172.21.15.6:0/3183945289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2f81064a0 0x7fc2f8108890 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00098a0 tx=0x7fc2f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95308e1378bf70b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:03:53.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.030+0000 7fc2f7fff700 1 -- 172.21.15.6:0/3183945289 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2f0004160 con 0x7fc2f81064a0 2024-03-24T12:03:53.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.031+0000 7fc2f7fff700 1 -- 172.21.15.6:0/3183945289 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f00042c0 con 0x7fc2f81064a0 2024-03-24T12:03:53.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.031+0000 7fc2f7fff700 1 -- 172.21.15.6:0/3183945289 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2f00045a0 con 0x7fc2f81064a0 2024-03-24T12:03:53.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.031+0000 7fc2ff2d6700 1 -- 172.21.15.6:0/3183945289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2f81064a0 msgr2=0x7fc2f8108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:03:53.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.031+0000 7fc2ff2d6700 1 --2- 172.21.15.6:0/3183945289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2f81064a0 0x7fc2f8108890 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00098a0 tx=0x7fc2f0009bb0 comp rx=0 tx=0).stop 2024-03-24T12:03:53.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.031+0000 7fc2ff2d6700 1 -- 172.21.15.6:0/3183945289 shutdown_connections 2024-03-24T12:03:53.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.031+0000 7fc2ff2d6700 1 --2- 172.21.15.6:0/3183945289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2f81064a0 0x7fc2f8108890 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:03:53.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.031+0000 7fc2ff2d6700 1 -- 172.21.15.6:0/3183945289 >> 172.21.15.6:0/3183945289 conn(0x7fc2f80ffef0 msgr2=0x7fc2f8102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:03:53.765 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.031+0000 7fc2ff2d6700 1 -- 172.21.15.6:0/3183945289 shutdown_connections 2024-03-24T12:03:53.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.031+0000 7fc2ff2d6700 1 -- 172.21.15.6:0/3183945289 wait complete. 2024-03-24T12:03:53.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.032+0000 7fc2ff2d6700 1 Processor -- start 2024-03-24T12:03:53.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.032+0000 7fc2ff2d6700 1 -- start start 2024-03-24T12:03:53.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.032+0000 7fc2ff2d6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2f81064a0 0x7fc2f8198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:03:53.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.032+0000 7fc2ff2d6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f81993c0 con 0x7fc2f81064a0 2024-03-24T12:03:53.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.032+0000 7fc2fd072700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2f81064a0 0x7fc2f8198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:03:53.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.032+0000 7fc2fd072700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2f81064a0 0x7fc2f8198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57050/0 (socket says 172.21.15.6:57050) 2024-03-24T12:03:53.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.032+0000 7fc2fd072700 1 -- 172.21.15.6:0/3606456592 learned_addr learned my addr 172.21.15.6:0/3606456592 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:03:53.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.032+0000 7fc2fd072700 1 -- 172.21.15.6:0/3606456592 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f0009550 con 0x7fc2f81064a0 2024-03-24T12:03:53.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.032+0000 7fc2fd072700 1 --2- 172.21.15.6:0/3606456592 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2f81064a0 0x7fc2f8198e80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8078b60 tx=0x7fc2f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:03:53.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.033+0000 7fc2f67fc700 1 -- 172.21.15.6:0/3606456592 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2f0011630 con 0x7fc2f81064a0 2024-03-24T12:03:53.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.033+0000 7fc2f67fc700 1 -- 172.21.15.6:0/3606456592 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f0011790 con 0x7fc2f81064a0 2024-03-24T12:03:53.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.033+0000 7fc2ff2d6700 1 -- 172.21.15.6:0/3606456592 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f81995c0 con 0x7fc2f81064a0 2024-03-24T12:03:53.766 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.033+0000 7fc2f67fc700 1 -- 172.21.15.6:0/3606456592 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2f0011a30 con 0x7fc2f81064a0 2024-03-24T12:03:53.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.033+0000 7fc2ff2d6700 1 -- 172.21.15.6:0/3606456592 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f8199a60 con 0x7fc2f81064a0 2024-03-24T12:03:53.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.033+0000 7fc2f67fc700 1 -- 172.21.15.6:0/3606456592 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc2f0022070 con 0x7fc2f81064a0 2024-03-24T12:03:53.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.033+0000 7fc2f67fc700 1 -- 172.21.15.6:0/3606456592 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc2f001c450 con 0x7fc2f81064a0 2024-03-24T12:03:53.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.034+0000 7fc2ff2d6700 1 -- 172.21.15.6:0/3606456592 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f8192cc0 con 0x7fc2f81064a0 2024-03-24T12:03:53.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.036+0000 7fc2f67fc700 1 -- 172.21.15.6:0/3606456592 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc2f0011b90 con 0x7fc2f81064a0 2024-03-24T12:03:53.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.108+0000 7fc2ff2d6700 1 -- 172.21.15.6:0/3606456592 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fc2f8109080 con 0x7fc2f81064a0 2024-03-24T12:03:53.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.111+0000 7fc2f67fc700 1 -- 172.21.15.6:0/3606456592 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+336 (secure 0 0 0) 0x7fc2f0011de0 con 0x7fc2f81064a0 2024-03-24T12:03:53.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.112+0000 7fc2f67fc700 1 -- 172.21.15.6:0/3606456592 <== mon.0 v2:172.21.15.6:3300/0 8 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fc2f001c6b0 con 0x7fc2f81064a0 2024-03-24T12:03:53.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.112+0000 7fc2ff2d6700 1 -- 172.21.15.6:0/3606456592 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2f81064a0 msgr2=0x7fc2f8198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:03:53.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.112+0000 7fc2ff2d6700 1 --2- 172.21.15.6:0/3606456592 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2f81064a0 0x7fc2f8198e80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8078b60 tx=0x7fc2f0004750 comp rx=0 tx=0).stop 2024-03-24T12:03:53.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.112+0000 7fc2ff2d6700 1 -- 172.21.15.6:0/3606456592 shutdown_connections 2024-03-24T12:03:53.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.112+0000 7fc2ff2d6700 1 --2- 172.21.15.6:0/3606456592 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2f81064a0 0x7fc2f8198e80 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:03:53.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.112+0000 7fc2ff2d6700 1 -- 172.21.15.6:0/3606456592 >> 172.21.15.6:0/3606456592 conn(0x7fc2f80ffef0 msgr2=0x7fc2f81021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:03:53.767 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.112+0000 7fc2ff2d6700 1 -- 172.21.15.6:0/3606456592 shutdown_connections 2024-03-24T12:03:53.768 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:50.112+0000 7fc2ff2d6700 1 -- 172.21.15.6:0/3606456592 wait complete. 2024-03-24T12:03:53.768 INFO:teuthology.orchestra.run.smithi006.stdout:Generating new minimal ceph.conf... 2024-03-24T12:03:54.584 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:54 smithi006 ceph-mon[29616]: from='client.? 172.21.15.6:0/3069953985' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:03:54.818 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.223+0000 7f1b28065700 1 Processor -- start 2024-03-24T12:03:54.818 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.223+0000 7f1b28065700 1 -- start start 2024-03-24T12:03:54.818 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.224+0000 7f1b28065700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b20104930 0x7f1b20104d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:03:54.818 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.224+0000 7f1b28065700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b20105320 con 0x7f1b20104930 2024-03-24T12:03:54.818 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.224+0000 7f1b25e01700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b20104930 0x7f1b20104d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:03:54.818 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.224+0000 7f1b25e01700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b20104930 0x7f1b20104d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57056/0 (socket says 172.21.15.6:57056) 2024-03-24T12:03:54.818 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.224+0000 7f1b25e01700 1 -- 172.21.15.6:0/865153562 learned_addr learned my addr 172.21.15.6:0/865153562 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:03:54.818 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.224+0000 7f1b25e01700 1 -- 172.21.15.6:0/865153562 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b20105b30 con 0x7f1b20104930 2024-03-24T12:03:54.818 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.224+0000 7f1b25e01700 1 --2- 172.21.15.6:0/865153562 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b20104930 0x7f1b20104d50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0098a0 tx=0x7f1b1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf16d7d7ed339de server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:03:54.818 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.224+0000 7f1b24dff700 1 -- 172.21.15.6:0/865153562 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b1c004160 con 0x7f1b20104930 2024-03-24T12:03:54.818 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.224+0000 7f1b24dff700 1 -- 172.21.15.6:0/865153562 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f1b1c0042c0 con 0x7f1b20104930 2024-03-24T12:03:54.819 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.224+0000 7f1b24dff700 1 -- 172.21.15.6:0/865153562 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b1c0118a0 con 0x7f1b20104930 2024-03-24T12:03:54.819 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.225+0000 7f1b28065700 1 -- 172.21.15.6:0/865153562 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b20104930 msgr2=0x7f1b20104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:03:54.819 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.225+0000 7f1b28065700 1 --2- 172.21.15.6:0/865153562 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b20104930 0x7f1b20104d50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0098a0 tx=0x7f1b1c009bb0 comp rx=0 tx=0).stop 2024-03-24T12:03:54.819 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.225+0000 7f1b28065700 1 -- 172.21.15.6:0/865153562 shutdown_connections 2024-03-24T12:03:54.819 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.225+0000 7f1b28065700 1 --2- 172.21.15.6:0/865153562 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b20104930 0x7f1b20104d50 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:03:54.819 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.225+0000 7f1b28065700 1 -- 172.21.15.6:0/865153562 >> 172.21.15.6:0/865153562 conn(0x7f1b200ffef0 msgr2=0x7f1b20102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:03:54.819 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.225+0000 7f1b28065700 1 -- 172.21.15.6:0/865153562 shutdown_connections 2024-03-24T12:03:54.819 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.225+0000 7f1b28065700 1 -- 172.21.15.6:0/865153562 wait complete. 2024-03-24T12:03:54.819 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.226+0000 7f1b28065700 1 Processor -- start 2024-03-24T12:03:54.819 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.226+0000 7f1b28065700 1 -- start start 2024-03-24T12:03:54.819 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.226+0000 7f1b28065700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b20104930 0x7f1b2018f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:03:54.819 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.226+0000 7f1b28065700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2018fe10 con 0x7f1b20104930 2024-03-24T12:03:54.819 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.226+0000 7f1b25e01700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b20104930 0x7f1b2018f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:03:54.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.226+0000 7f1b25e01700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b20104930 0x7f1b2018f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57060/0 (socket says 172.21.15.6:57060) 2024-03-24T12:03:54.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.226+0000 7f1b25e01700 1 -- 172.21.15.6:0/3069953985 learned_addr learned my addr 172.21.15.6:0/3069953985 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:03:54.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.227+0000 7f1b25e01700 1 -- 172.21.15.6:0/3069953985 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1c009550 con 0x7f1b20104930 2024-03-24T12:03:54.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.227+0000 7f1b25e01700 1 --2- 172.21.15.6:0/3069953985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b20104930 0x7f1b2018f8d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c003f10 tx=0x7f1b1c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:03:54.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.227+0000 7f1b16ffd700 1 -- 172.21.15.6:0/3069953985 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b1c0036a0 con 0x7f1b20104930 2024-03-24T12:03:54.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.227+0000 7f1b16ffd700 1 -- 172.21.15.6:0/3069953985 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f1b1c01a430 con 0x7f1b20104930 2024-03-24T12:03:54.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.227+0000 7f1b16ffd700 1 -- 172.21.15.6:0/3069953985 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b1c024410 con 0x7f1b20104930 2024-03-24T12:03:54.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.227+0000 7f1b28065700 1 -- 172.21.15.6:0/3069953985 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b2018e000 con 0x7f1b20104930 2024-03-24T12:03:54.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.227+0000 7f1b28065700 1 -- 172.21.15.6:0/3069953985 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b2018e4a0 con 0x7f1b20104930 2024-03-24T12:03:54.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.227+0000 7f1b16ffd700 1 -- 172.21.15.6:0/3069953985 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f1b1c024a10 con 0x7f1b20104930 2024-03-24T12:03:54.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.227+0000 7f1b16ffd700 1 -- 172.21.15.6:0/3069953985 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1b1c0193c0 con 0x7f1b20104930 2024-03-24T12:03:54.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.228+0000 7f1b28065700 1 -- 172.21.15.6:0/3069953985 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b20106870 con 0x7f1b20104930 2024-03-24T12:03:54.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.230+0000 7f1b16ffd700 1 -- 172.21.15.6:0/3069953985 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f1b1c02de40 con 0x7f1b20104930 2024-03-24T12:03:54.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.300+0000 7f1b28065700 1 -- 172.21.15.6:0/3069953985 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f1b201931e0 con 0x7f1b20104930 2024-03-24T12:03:54.820 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.301+0000 7f1b16ffd700 1 -- 172.21.15.6:0/3069953985 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+173 (secure 0 0 0) 0x7f1b1c01a5a0 con 0x7f1b20104930 2024-03-24T12:03:54.821 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.301+0000 7f1b28065700 1 -- 172.21.15.6:0/3069953985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b20104930 msgr2=0x7f1b2018f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:03:54.821 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.301+0000 7f1b28065700 1 --2- 172.21.15.6:0/3069953985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b20104930 0x7f1b2018f8d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c003f10 tx=0x7f1b1c011940 comp rx=0 tx=0).stop 2024-03-24T12:03:54.821 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.301+0000 7f1b28065700 1 -- 172.21.15.6:0/3069953985 shutdown_connections 2024-03-24T12:03:54.821 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.301+0000 7f1b28065700 1 --2- 172.21.15.6:0/3069953985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b20104930 0x7f1b2018f8d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:03:54.821 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.301+0000 7f1b28065700 1 -- 172.21.15.6:0/3069953985 >> 172.21.15.6:0/3069953985 conn(0x7f1b200ffef0 msgr2=0x7f1b20100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:03:54.821 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.302+0000 7f1b28065700 1 -- 172.21.15.6:0/3069953985 shutdown_connections 2024-03-24T12:03:54.821 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:54.302+0000 7f1b28065700 1 -- 172.21.15.6:0/3069953985 wait complete. 2024-03-24T12:03:54.821 INFO:teuthology.orchestra.run.smithi006.stdout:Restarting the monitor... 2024-03-24T12:03:55.093 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:54 smithi006 systemd[1]: Stopping Ceph mon.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:03:55.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-a[29593]: 2024-03-24T12:03:55.091+0000 7fe71d6f2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T12:03:55.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-a[29593]: 2024-03-24T12:03:55.091+0000 7fe71d6f2700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-24T12:03:55.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:55 smithi006 podman[29949]: 2024-03-24 12:03:55.234890584 +0000 UTC m=+0.249511797 container died 1622c4564db706cae62daf3c2e20c66faf71ff973a193657641332bbde716c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-a, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T12:03:55.838 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:55 smithi006 podman[29949]: 2024-03-24 12:03:55.605435278 +0000 UTC m=+0.620056502 container cleanup 1622c4564db706cae62daf3c2e20c66faf71ff973a193657641332bbde716c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-a, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T12:03:55.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:55 smithi006 bash[29949]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-a 2024-03-24T12:03:56.181 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:55 smithi006 podman[29968]: 2024-03-24 12:03:55.839920562 +0000 UTC m=+0.608518362 container remove 1622c4564db706cae62daf3c2e20c66faf71ff973a193657641332bbde716c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-03-24T12:03:56.181 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:55 smithi006 systemd[1]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-24T12:03:56.182 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:55 smithi006 systemd[1]: Stopped Ceph mon.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:03:56.182 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:55 smithi006 systemd[1]: Starting Ceph mon.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:03:56.442 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 podman[30070]: 2024-03-24 12:03:56.182676672 +0000 UTC m=+0.109086966 container create dcf24dc5c2fcfa3c3f11afa79ccfc9a7fb55ce39eb84d228ea0714052441be44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-24T12:03:56.442 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 podman[30070]: 2024-03-24 12:03:56.091290289 +0000 UTC m=+0.017700534 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T12:03:56.442 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 podman[30070]: 2024-03-24 12:03:56.368323977 +0000 UTC m=+0.294734225 container init dcf24dc5c2fcfa3c3f11afa79ccfc9a7fb55ce39eb84d228ea0714052441be44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-a, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-03-24T12:03:56.442 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 podman[30070]: 2024-03-24 12:03:56.373944676 +0000 UTC m=+0.300354922 container start dcf24dc5c2fcfa3c3f11afa79ccfc9a7fb55ce39eb84d228ea0714052441be44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-24T12:03:56.443 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: set uid:gid to 167:167 (ceph:ceph) 2024-03-24T12:03:56.443 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev), process ceph-mon, pid 7 2024-03-24T12:03:56.443 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: pidfile_write: ignore empty --pid-file 2024-03-24T12:03:56.443 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: load: jerasure load: lrc 2024-03-24T12:03:56.443 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: RocksDB version: 7.9.2 2024-03-24T12:03:56.443 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Git sha 0 2024-03-24T12:03:56.443 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Compile date 2024-03-16 10:36:11 2024-03-24T12:03:56.443 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: DB SUMMARY 2024-03-24T12:03:56.443 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: DB Session ID: LFQQUWI0TUPJAP1BP7ZJ 2024-03-24T12:03:56.443 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: CURRENT file: CURRENT 2024-03-24T12:03:56.443 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: IDENTITY file: IDENTITY 2024-03-24T12:03:56.443 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-24T12:03:56.444 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-24T12:03:56.444 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 86791 ; 2024-03-24T12:03:56.444 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.error_if_exists: 0 2024-03-24T12:03:56.445 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.create_if_missing: 0 2024-03-24T12:03:56.445 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.paranoid_checks: 1 2024-03-24T12:03:56.445 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-24T12:03:56.445 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-24T12:03:56.445 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-24T12:03:56.445 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.env: 0x5621d57daae0 2024-03-24T12:03:56.445 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.fs: PosixFileSystem 2024-03-24T12:03:56.445 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.info_log: 0x5621d75618c0 2024-03-24T12:03:56.445 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_file_opening_threads: 16 2024-03-24T12:03:56.446 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.statistics: (nil) 2024-03-24T12:03:56.446 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.use_fsync: 0 2024-03-24T12:03:56.446 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_log_file_size: 0 2024-03-24T12:03:56.446 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-24T12:03:56.446 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-24T12:03:56.446 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.keep_log_file_num: 1000 2024-03-24T12:03:56.446 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.recycle_log_file_num: 0 2024-03-24T12:03:56.446 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.allow_fallocate: 1 2024-03-24T12:03:56.446 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.allow_mmap_reads: 0 2024-03-24T12:03:56.446 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.allow_mmap_writes: 0 2024-03-24T12:03:56.446 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.use_direct_reads: 0 2024-03-24T12:03:56.446 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-24T12:03:56.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.create_missing_column_families: 0 2024-03-24T12:03:56.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.db_log_dir: 2024-03-24T12:03:56.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.wal_dir: 2024-03-24T12:03:56.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-24T12:03:56.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-24T12:03:56.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-24T12:03:56.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-24T12:03:56.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-24T12:03:56.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-24T12:03:56.447 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.advise_random_on_open: 1 2024-03-24T12:03:56.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.db_write_buffer_size: 0 2024-03-24T12:03:56.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.write_buffer_manager: 0x5621d7535400 2024-03-24T12:03:56.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-24T12:03:56.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-24T12:03:56.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-24T12:03:56.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.rate_limiter: (nil) 2024-03-24T12:03:56.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-24T12:03:56.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.wal_recovery_mode: 2 2024-03-24T12:03:56.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.enable_thread_tracking: 0 2024-03-24T12:03:56.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.enable_pipelined_write: 0 2024-03-24T12:03:56.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.unordered_write: 0 2024-03-24T12:03:56.448 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-24T12:03:56.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-24T12:03:56.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-24T12:03:56.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-24T12:03:56.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.row_cache: None 2024-03-24T12:03:56.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.wal_filter: None 2024-03-24T12:03:56.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-24T12:03:56.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.allow_ingest_behind: 0 2024-03-24T12:03:56.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.two_write_queues: 0 2024-03-24T12:03:56.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.manual_wal_flush: 0 2024-03-24T12:03:56.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.wal_compression: 0 2024-03-24T12:03:56.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.atomic_flush: 0 2024-03-24T12:03:56.449 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-24T12:03:56.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-24T12:03:56.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-24T12:03:56.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.log_readahead_size: 0 2024-03-24T12:03:56.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-24T12:03:56.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.best_efforts_recovery: 0 2024-03-24T12:03:56.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-24T12:03:56.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-24T12:03:56.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.allow_data_in_errors: 0 2024-03-24T12:03:56.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.db_host_id: __hostname__ 2024-03-24T12:03:56.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-24T12:03:56.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_background_jobs: 2 2024-03-24T12:03:56.450 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_background_compactions: -1 2024-03-24T12:03:56.451 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_subcompactions: 1 2024-03-24T12:03:56.451 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-24T12:03:56.451 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-24T12:03:56.451 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-24T12:03:56.451 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_total_wal_size: 0 2024-03-24T12:03:56.451 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-24T12:03:56.451 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-24T12:03:56.451 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-24T12:03:56.451 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-24T12:03:56.451 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_open_files: -1 2024-03-24T12:03:56.451 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.bytes_per_sync: 0 2024-03-24T12:03:56.451 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-24T12:03:56.452 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-24T12:03:56.452 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.compaction_readahead_size: 0 2024-03-24T12:03:56.452 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_background_flushes: -1 2024-03-24T12:03:56.452 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Compression algorithms supported: 2024-03-24T12:03:56.452 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-24T12:03:56.452 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: kZSTD supported: 0 2024-03-24T12:03:56.452 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: kXpressCompression supported: 0 2024-03-24T12:03:56.452 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: kLZ4HCCompression supported: 1 2024-03-24T12:03:56.452 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: kZlibCompression supported: 1 2024-03-24T12:03:56.452 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: kSnappyCompression supported: 1 2024-03-24T12:03:56.452 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: kLZ4Compression supported: 1 2024-03-24T12:03:56.453 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: kBZip2Compression supported: 0 2024-03-24T12:03:56.453 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-24T12:03:56.453 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-24T12:03:56.453 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-24T12:03:56.453 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-24T12:03:56.453 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-24T12:03:56.453 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.merge_operator: 2024-03-24T12:03:56.453 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.compaction_filter: None 2024-03-24T12:03:56.453 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.compaction_filter_factory: None 2024-03-24T12:03:56.453 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.sst_partitioner_factory: None 2024-03-24T12:03:56.453 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-24T12:03:56.453 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-24T12:03:56.453 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5621d75619c0) 2024-03-24T12:03:56.454 INFO:journalctl@ceph.mon.a.smithi006.stdout: cache_index_and_filter_blocks: 1 2024-03-24T12:03:56.454 INFO:journalctl@ceph.mon.a.smithi006.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-24T12:03:56.454 INFO:journalctl@ceph.mon.a.smithi006.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-24T12:03:56.454 INFO:journalctl@ceph.mon.a.smithi006.stdout: pin_top_level_index_and_filter: 1 2024-03-24T12:03:56.454 INFO:journalctl@ceph.mon.a.smithi006.stdout: index_type: 0 2024-03-24T12:03:56.454 INFO:journalctl@ceph.mon.a.smithi006.stdout: data_block_index_type: 0 2024-03-24T12:03:56.454 INFO:journalctl@ceph.mon.a.smithi006.stdout: index_shortening: 1 2024-03-24T12:03:56.454 INFO:journalctl@ceph.mon.a.smithi006.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-24T12:03:56.454 INFO:journalctl@ceph.mon.a.smithi006.stdout: checksum: 4 2024-03-24T12:03:56.454 INFO:journalctl@ceph.mon.a.smithi006.stdout: no_block_cache: 0 2024-03-24T12:03:56.454 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_cache: 0x5621d7565090 2024-03-24T12:03:56.455 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_cache_name: BinnedLRUCache 2024-03-24T12:03:56.455 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_cache_options: 2024-03-24T12:03:56.455 INFO:journalctl@ceph.mon.a.smithi006.stdout: capacity : 536870912 2024-03-24T12:03:56.455 INFO:journalctl@ceph.mon.a.smithi006.stdout: num_shard_bits : 4 2024-03-24T12:03:56.455 INFO:journalctl@ceph.mon.a.smithi006.stdout: strict_capacity_limit : 0 2024-03-24T12:03:56.455 INFO:journalctl@ceph.mon.a.smithi006.stdout: high_pri_pool_ratio: 0.000 2024-03-24T12:03:56.455 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_cache_compressed: (nil) 2024-03-24T12:03:56.455 INFO:journalctl@ceph.mon.a.smithi006.stdout: persistent_cache: (nil) 2024-03-24T12:03:56.455 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_size: 4096 2024-03-24T12:03:56.455 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_size_deviation: 10 2024-03-24T12:03:56.455 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_restart_interval: 16 2024-03-24T12:03:56.456 INFO:journalctl@ceph.mon.a.smithi006.stdout: index_block_restart_interval: 1 2024-03-24T12:03:56.456 INFO:journalctl@ceph.mon.a.smithi006.stdout: metadata_block_size: 4096 2024-03-24T12:03:56.456 INFO:journalctl@ceph.mon.a.smithi006.stdout: partition_filters: 0 2024-03-24T12:03:56.456 INFO:journalctl@ceph.mon.a.smithi006.stdout: use_delta_encoding: 1 2024-03-24T12:03:56.456 INFO:journalctl@ceph.mon.a.smithi006.stdout: filter_policy: bloomfilter 2024-03-24T12:03:56.456 INFO:journalctl@ceph.mon.a.smithi006.stdout: whole_key_filtering: 1 2024-03-24T12:03:56.456 INFO:journalctl@ceph.mon.a.smithi006.stdout: verify_compression: 0 2024-03-24T12:03:56.456 INFO:journalctl@ceph.mon.a.smithi006.stdout: read_amp_bytes_per_bit: 0 2024-03-24T12:03:56.456 INFO:journalctl@ceph.mon.a.smithi006.stdout: format_version: 5 2024-03-24T12:03:56.456 INFO:journalctl@ceph.mon.a.smithi006.stdout: enable_index_compression: 1 2024-03-24T12:03:56.456 INFO:journalctl@ceph.mon.a.smithi006.stdout: block_align: 0 2024-03-24T12:03:56.457 INFO:journalctl@ceph.mon.a.smithi006.stdout: max_auto_readahead_size: 262144 2024-03-24T12:03:56.457 INFO:journalctl@ceph.mon.a.smithi006.stdout: prepopulate_block_cache: 0 2024-03-24T12:03:56.457 INFO:journalctl@ceph.mon.a.smithi006.stdout: initial_auto_readahead_size: 8192 2024-03-24T12:03:56.457 INFO:journalctl@ceph.mon.a.smithi006.stdout: num_file_reads_for_auto_readahead: 2 2024-03-24T12:03:56.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.write_buffer_size: 33554432 2024-03-24T12:03:56.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_write_buffer_number: 2 2024-03-24T12:03:56.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.compression: NoCompression 2024-03-24T12:03:56.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.bottommost_compression: Disabled 2024-03-24T12:03:56.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.prefix_extractor: nullptr 2024-03-24T12:03:56.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-24T12:03:56.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.num_levels: 7 2024-03-24T12:03:56.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-24T12:03:56.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-24T12:03:56.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-24T12:03:56.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-24T12:03:56.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-24T12:03:56.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-24T12:03:56.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-24T12:03:56.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-24T12:03:56.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-24T12:03:56.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-24T12:03:56.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-24T12:03:56.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-24T12:03:56.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-24T12:03:56.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.compression_opts.level: 32767 2024-03-24T12:03:56.459 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.compression_opts.strategy: 0 2024-03-24T12:03:56.459 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-24T12:03:56.459 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-24T12:03:56.459 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-24T12:03:56.459 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-24T12:03:56.459 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.compression_opts.enabled: false 2024-03-24T12:03:56.459 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-24T12:03:56.459 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-24T12:03:56.460 INFO:teuthology.orchestra.run.smithi006.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-24T12:03:56.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-24T12:03:56.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-24T12:03:56.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.target_file_size_base: 67108864 2024-03-24T12:03:56.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-24T12:03:56.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-24T12:03:56.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-24T12:03:56.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-24T12:03:56.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-24T12:03:56.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-24T12:03:56.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-24T12:03:56.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-24T12:03:56.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-24T12:03:56.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-24T12:03:56.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-24T12:03:56.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-24T12:03:56.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-24T12:03:56.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-24T12:03:56.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.arena_block_size: 1048576 2024-03-24T12:03:56.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-24T12:03:56.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-24T12:03:56.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.disable_auto_compactions: 0 2024-03-24T12:03:56.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-24T12:03:56.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-24T12:03:56.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-24T12:03:56.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-24T12:03:56.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-24T12:03:56.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-24T12:03:56.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-24T12:03:56.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-24T12:03:56.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-24T12:03:56.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-24T12:03:56.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.table_properties_collectors: 2024-03-24T12:03:56.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.inplace_update_support: 0 2024-03-24T12:03:56.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-24T12:03:56.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-24T12:03:56.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-24T12:03:56.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-24T12:03:56.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.bloom_locality: 0 2024-03-24T12:03:56.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.max_successive_merges: 0 2024-03-24T12:03:56.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-24T12:03:56.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.paranoid_file_checks: 0 2024-03-24T12:03:56.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.force_consistency_checks: 1 2024-03-24T12:03:56.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.report_bg_io_stats: 0 2024-03-24T12:03:56.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.ttl: 2592000 2024-03-24T12:03:56.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-24T12:03:56.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-24T12:03:56.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-24T12:03:56.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.enable_blob_files: false 2024-03-24T12:03:56.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.min_blob_size: 0 2024-03-24T12:03:56.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.blob_file_size: 268435456 2024-03-24T12:03:56.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-24T12:03:56.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-24T12:03:56.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-24T12:03:56.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-24T12:03:56.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-24T12:03:56.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.blob_file_starting_level: 0 2024-03-24T12:03:56.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-24T12:03:56.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-24T12:03:56.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-24T12:03:56.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0977c70c-128b-46d8-91f5-fd2087f2d237 2024-03-24T12:03:56.466 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711281836402242, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-24T12:03:56.466 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-24T12:03:56.466 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711281836403097, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 83213, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 262, "table_properties": {"data_size": 81385, "index_size": 238, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10592, "raw_average_key_size": 47, "raw_value_size": 75397, "raw_average_value_size": 341, "num_data_blocks": 10, "num_entries": 221, "num_filter_entries": 221, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711281836, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0977c70c-128b-46d8-91f5-fd2087f2d237", "db_session_id": "LFQQUWI0TUPJAP1BP7ZJ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-24T12:03:56.466 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711281836403181, "job": 1, "event": "recovery_finished"} 2024-03-24T12:03:56.466 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-24T12:03:56.466 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-24T12:03:56.466 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5621d7656000 2024-03-24T12:03:56.466 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: DB pointer 0x5621d7640000 2024-03-24T12:03:56.466 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-24T12:03:56.466 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-24T12:03:56.466 INFO:journalctl@ceph.mon.a.smithi006.stdout: ** DB Stats ** 2024-03-24T12:03:56.467 INFO:journalctl@ceph.mon.a.smithi006.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T12:03:56.467 INFO:journalctl@ceph.mon.a.smithi006.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-24T12:03:56.467 INFO:journalctl@ceph.mon.a.smithi006.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T12:03:56.467 INFO:journalctl@ceph.mon.a.smithi006.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T12:03:56.467 INFO:journalctl@ceph.mon.a.smithi006.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-24T12:03:56.467 INFO:journalctl@ceph.mon.a.smithi006.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T12:03:56.467 INFO:journalctl@ceph.mon.a.smithi006.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T12:03:56.467 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-03-24T12:03:56.467 INFO:journalctl@ceph.mon.a.smithi006.stdout: ** Compaction Stats [default] ** 2024-03-24T12:03:56.467 INFO:journalctl@ceph.mon.a.smithi006.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T12:03:56.467 INFO:journalctl@ceph.mon.a.smithi006.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-24T12:03:56.467 INFO:journalctl@ceph.mon.a.smithi006.stdout: L0 2/0 83.10 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-24T12:03:56.467 INFO:journalctl@ceph.mon.a.smithi006.stdout: Sum 2/0 83.10 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-24T12:03:56.468 INFO:journalctl@ceph.mon.a.smithi006.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-24T12:03:56.468 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-03-24T12:03:56.468 INFO:journalctl@ceph.mon.a.smithi006.stdout: ** Compaction Stats [default] ** 2024-03-24T12:03:56.468 INFO:journalctl@ceph.mon.a.smithi006.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T12:03:56.468 INFO:journalctl@ceph.mon.a.smithi006.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T12:03:56.468 INFO:journalctl@ceph.mon.a.smithi006.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 144.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-24T12:03:56.468 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-03-24T12:03:56.468 INFO:journalctl@ceph.mon.a.smithi006.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-24T12:03:56.468 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-03-24T12:03:56.468 INFO:journalctl@ceph.mon.a.smithi006.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T12:03:56.468 INFO:journalctl@ceph.mon.a.smithi006.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-24T12:03:56.468 INFO:journalctl@ceph.mon.a.smithi006.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-24T12:03:56.468 INFO:journalctl@ceph.mon.a.smithi006.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-24T12:03:56.469 INFO:journalctl@ceph.mon.a.smithi006.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-24T12:03:56.469 INFO:journalctl@ceph.mon.a.smithi006.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-24T12:03:56.469 INFO:journalctl@ceph.mon.a.smithi006.stdout: Cumulative compaction: 0.00 GB write, 27.84 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T12:03:56.469 INFO:journalctl@ceph.mon.a.smithi006.stdout: Interval compaction: 0.00 GB write, 27.84 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T12:03:56.469 INFO:journalctl@ceph.mon.a.smithi006.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-24T12:03:56.469 INFO:journalctl@ceph.mon.a.smithi006.stdout: Block cache BinnedLRUCache@0x5621d7565090#7 capacity: 512.00 MB usage: 1.19 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-24T12:03:56.469 INFO:journalctl@ceph.mon.a.smithi006.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.42 KB,8.04663e-05%) Misc(1,0.00 KB,0%) 2024-03-24T12:03:56.469 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-03-24T12:03:56.469 INFO:journalctl@ceph.mon.a.smithi006.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-24T12:03:56.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: starting mon.a rank 0 at public addrs [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] at bind addrs [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 2024-03-24T12:03:56.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: mon.a@-1(???) e1 preinit fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 2024-03-24T12:03:56.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: mon.a@-1(???).mds e1 new map 2024-03-24T12:03:56.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: mon.a@-1(???).mds e1 print_map 2024-03-24T12:03:56.470 INFO:journalctl@ceph.mon.a.smithi006.stdout: e1 2024-03-24T12:03:56.470 INFO:journalctl@ceph.mon.a.smithi006.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T12:03:56.470 INFO:journalctl@ceph.mon.a.smithi006.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-24T12:03:56.470 INFO:journalctl@ceph.mon.a.smithi006.stdout: legacy client fscid: -1 2024-03-24T12:03:56.470 INFO:journalctl@ceph.mon.a.smithi006.stdout: 2024-03-24T12:03:56.470 INFO:journalctl@ceph.mon.a.smithi006.stdout: No filesystems configured 2024-03-24T12:03:56.470 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-24T12:03:56.470 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T12:03:56.470 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T12:03:56.470 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T12:03:56.470 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-24T12:03:56.470 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 bash[30070]: dcf24dc5c2fcfa3c3f11afa79ccfc9a7fb55ce39eb84d228ea0714052441be44 2024-03-24T12:03:56.471 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 systemd[1]: Started Ceph mon.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:03:56.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-24T12:03:56.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: monmap e1: 1 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T12:03:56.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: fsmap 2024-03-24T12:03:56.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: osdmap e1: 0 total, 0 up, 0 in 2024-03-24T12:03:56.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:56 smithi006 ceph-mon[30108]: mgrmap e1: no daemons active 2024-03-24T12:03:57.564 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.927+0000 7f8a6b726700 1 Processor -- start 2024-03-24T12:03:57.564 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.927+0000 7f8a6b726700 1 -- start start 2024-03-24T12:03:57.564 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.928+0000 7f8a6b726700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a64079b90 0x7f8a64079fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:03:57.564 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.928+0000 7f8a6b726700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6407b8c0 con 0x7f8a64079b90 2024-03-24T12:03:57.564 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.928+0000 7f8a694c2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a64079b90 0x7f8a64079fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:03:57.564 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.928+0000 7f8a694c2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a64079b90 0x7f8a64079fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59312/0 (socket says 172.21.15.6:59312) 2024-03-24T12:03:57.564 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.928+0000 7f8a694c2700 1 -- 172.21.15.6:0/281165538 learned_addr learned my addr 172.21.15.6:0/281165538 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:03:57.564 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.928+0000 7f8a694c2700 1 -- 172.21.15.6:0/281165538 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6407a4f0 con 0x7f8a64079b90 2024-03-24T12:03:57.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.928+0000 7f8a694c2700 1 --2- 172.21.15.6:0/281165538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a64079b90 0x7f8a64079fb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8a600098a0 tx=0x7f8a60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce959072402e1a server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:03:57.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.929+0000 7f8a5bfff700 1 -- 172.21.15.6:0/281165538 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a60004160 con 0x7f8a64079b90 2024-03-24T12:03:57.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.929+0000 7f8a5bfff700 1 -- 172.21.15.6:0/281165538 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f8a600042c0 con 0x7f8a64079b90 2024-03-24T12:03:57.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.929+0000 7f8a5bfff700 1 -- 172.21.15.6:0/281165538 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a600118a0 con 0x7f8a64079b90 2024-03-24T12:03:57.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.929+0000 7f8a6b726700 1 -- 172.21.15.6:0/281165538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a64079b90 msgr2=0x7f8a64079fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:03:57.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.929+0000 7f8a6b726700 1 --2- 172.21.15.6:0/281165538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a64079b90 0x7f8a64079fb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8a600098a0 tx=0x7f8a60009bb0 comp rx=0 tx=0).stop 2024-03-24T12:03:57.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.929+0000 7f8a6b726700 1 -- 172.21.15.6:0/281165538 shutdown_connections 2024-03-24T12:03:57.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.929+0000 7f8a6b726700 1 --2- 172.21.15.6:0/281165538 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a64079b90 0x7f8a64079fb0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:03:57.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.929+0000 7f8a6b726700 1 -- 172.21.15.6:0/281165538 >> 172.21.15.6:0/281165538 conn(0x7f8a64100fa0 msgr2=0x7f8a64103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:03:57.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.929+0000 7f8a6b726700 1 -- 172.21.15.6:0/281165538 shutdown_connections 2024-03-24T12:03:57.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.929+0000 7f8a6b726700 1 -- 172.21.15.6:0/281165538 wait complete. 2024-03-24T12:03:57.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.930+0000 7f8a6b726700 1 Processor -- start 2024-03-24T12:03:57.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.930+0000 7f8a6b726700 1 -- start start 2024-03-24T12:03:57.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.930+0000 7f8a6b726700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a64079b90 0x7f8a6419b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:03:57.566 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.930+0000 7f8a6b726700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6419bea0 con 0x7f8a64079b90 2024-03-24T12:03:57.566 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.930+0000 7f8a694c2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a64079b90 0x7f8a6419b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:03:57.566 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.930+0000 7f8a694c2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a64079b90 0x7f8a6419b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59326/0 (socket says 172.21.15.6:59326) 2024-03-24T12:03:57.566 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.930+0000 7f8a694c2700 1 -- 172.21.15.6:0/804397211 learned_addr learned my addr 172.21.15.6:0/804397211 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:03:57.566 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.930+0000 7f8a694c2700 1 -- 172.21.15.6:0/804397211 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a60009550 con 0x7f8a64079b90 2024-03-24T12:03:57.566 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.931+0000 7f8a694c2700 1 --2- 172.21.15.6:0/804397211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a64079b90 0x7f8a6419b960 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8a60003da0 tx=0x7f8a60011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:03:57.566 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.931+0000 7f8a5a7fc700 1 -- 172.21.15.6:0/804397211 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a60011930 con 0x7f8a64079b90 2024-03-24T12:03:57.566 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.931+0000 7f8a6b726700 1 -- 172.21.15.6:0/804397211 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a6419c0a0 con 0x7f8a64079b90 2024-03-24T12:03:57.566 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.931+0000 7f8a5a7fc700 1 -- 172.21.15.6:0/804397211 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f8a60003b00 con 0x7f8a64079b90 2024-03-24T12:03:57.566 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.931+0000 7f8a5a7fc700 1 -- 172.21.15.6:0/804397211 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a6001ad50 con 0x7f8a64079b90 2024-03-24T12:03:57.566 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.931+0000 7f8a6b726700 1 -- 172.21.15.6:0/804397211 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a6419c540 con 0x7f8a64079b90 2024-03-24T12:03:57.566 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.931+0000 7f8a5a7fc700 1 -- 172.21.15.6:0/804397211 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8a6002d7d0 con 0x7f8a64079b90 2024-03-24T12:03:57.566 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.931+0000 7f8a5a7fc700 1 -- 172.21.15.6:0/804397211 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8a60024730 con 0x7f8a64079b90 2024-03-24T12:03:57.567 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.932+0000 7f8a6b726700 1 -- 172.21.15.6:0/804397211 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a6410bdf0 con 0x7f8a64079b90 2024-03-24T12:03:57.567 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:56.934+0000 7f8a5a7fc700 1 -- 172.21.15.6:0/804397211 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8a60019db0 con 0x7f8a64079b90 2024-03-24T12:03:57.567 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:57.003+0000 7f8a6b726700 1 -- 172.21.15.6:0/804397211 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f8a6419ee90 con 0x7f8a64079b90 2024-03-24T12:03:57.567 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:57.005+0000 7f8a5a7fc700 1 -- 172.21.15.6:0/804397211 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8a6001a630 con 0x7f8a64079b90 2024-03-24T12:03:57.567 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:57.006+0000 7f8a6b726700 1 -- 172.21.15.6:0/804397211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a64079b90 msgr2=0x7f8a6419b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:03:57.567 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:57.006+0000 7f8a6b726700 1 --2- 172.21.15.6:0/804397211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a64079b90 0x7f8a6419b960 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8a60003da0 tx=0x7f8a60011e90 comp rx=0 tx=0).stop 2024-03-24T12:03:57.567 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:57.006+0000 7f8a6b726700 1 -- 172.21.15.6:0/804397211 shutdown_connections 2024-03-24T12:03:57.567 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:57.006+0000 7f8a6b726700 1 --2- 172.21.15.6:0/804397211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a64079b90 0x7f8a6419b960 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:03:57.567 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:57.006+0000 7f8a6b726700 1 -- 172.21.15.6:0/804397211 >> 172.21.15.6:0/804397211 conn(0x7f8a64100fa0 msgr2=0x7f8a64101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:03:57.567 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:57.006+0000 7f8a6b726700 1 -- 172.21.15.6:0/804397211 shutdown_connections 2024-03-24T12:03:57.567 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:57.006+0000 7f8a6b726700 1 -- 172.21.15.6:0/804397211 wait complete. 2024-03-24T12:03:57.567 INFO:teuthology.orchestra.run.smithi006.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-24T12:03:57.590 INFO:teuthology.orchestra.run.smithi006.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-24T12:03:57.590 INFO:teuthology.orchestra.run.smithi006.stdout:Creating mgr... 2024-03-24T12:03:57.590 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-24T12:03:57.591 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-24T12:03:57.840 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mgr.y 2024-03-24T12:03:57.840 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Failed to reset failed state of unit ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mgr.y.service: Unit ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mgr.y.service not loaded. 2024-03-24T12:03:57.988 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715.target.wants/ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mgr.y.service → /etc/systemd/system/ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@.service. 2024-03-24T12:03:58.315 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:58 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/804397211' entity='client.admin' 2024-03-24T12:03:58.315 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:03:58 smithi006 podman[30347]: 2024-03-24 12:03:58.129488741 +0000 UTC m=+0.017104174 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-24T12:03:58.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:03:58 smithi006 podman[30347]: 2024-03-24 12:03:58.437569205 +0000 UTC m=+0.325184630 container init efc19a5fffa4bc3c518f0109b506f1f2e76337d40b33778a31b691284786b4fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-24T12:03:58.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:03:58 smithi006 podman[30347]: 2024-03-24 12:03:58.442839619 +0000 UTC m=+0.330455045 container start efc19a5fffa4bc3c518f0109b506f1f2e76337d40b33778a31b691284786b4fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-24T12:03:58.800 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-24T12:03:58.800 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2024-03-24T12:03:58.808 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-24T12:03:58.808 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-03-24T12:03:58.808 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2024-03-24T12:03:58.808 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to enable service . firewalld.service is not available 2024-03-24T12:03:58.817 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-24T12:03:58.817 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2024-03-24T12:03:58.824 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-24T12:03:58.824 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-03-24T12:03:58.824 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2024-03-24T12:03:58.824 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-24T12:03:58.824 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr to start... 2024-03-24T12:03:58.824 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr... 2024-03-24T12:03:59.107 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:03:58 smithi006 bash[30347]: efc19a5fffa4bc3c518f0109b506f1f2e76337d40b33778a31b691284786b4fb 2024-03-24T12:03:59.107 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:03:58 smithi006 systemd[1]: Started Ceph mgr.y for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:03:59.107 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:03:59 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:03:59.011+0000 7f3c8ce27200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T12:03:59.107 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:03:59 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:03:59.107+0000 7f3c8ce27200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T12:03:59.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:03:59 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1508659535' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-24T12:03:59.961 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:03:59 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:03:59.792+0000 7f3c8ce27200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T12:03:59.961 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:03:59 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:03:59.887+0000 7f3c8ce27200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T12:04:00.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-24T12:04:00.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-03-24T12:04:00.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsid": "66d0ad74-e9d6-11ee-95cd-87774f69a715", 2024-03-24T12:04:00.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "health": { 2024-03-24T12:04:00.220 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-24T12:04:00.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-24T12:04:00.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-24T12:04:00.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:00.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-24T12:04:00.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-24T12:04:00.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 0 2024-03-24T12:04:00.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-24T12:04:00.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-24T12:04:00.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "a" 2024-03-24T12:04:00.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-24T12:04:00.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-03-24T12:04:00.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-24T12:04:00.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T12:04:00.221 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-24T12:04:00.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-24T12:04:00.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:00.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-24T12:04:00.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T12:04:00.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-24T12:04:00.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-24T12:04:00.222 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-24T12:04:00.230 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-24T12:04:00.230 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-24T12:04:00.230 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-24T12:04:00.230 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:00.230 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-24T12:04:00.230 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-24T12:04:00.230 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-24T12:04:00.230 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-24T12:04:00.230 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-24T12:04:00.230 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-24T12:04:00.230 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-24T12:04:00.230 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-24T12:04:00.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-24T12:04:00.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:00.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-24T12:04:00.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T12:04:00.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-24T12:04:00.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-24T12:04:00.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:00.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-24T12:04:00.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-24T12:04:00.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-24T12:04:00.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-24T12:04:00.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-24T12:04:00.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-24T12:04:00.231 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "restful" 2024-03-24T12:04:00.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-24T12:04:00.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T12:04:00.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:00.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-24T12:04:00.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T12:04:00.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modified": "2024-03-24T12:03:42.337658+0000", 2024-03-24T12:04:00.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T12:04:00.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:00.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-24T12:04:00.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-03-24T12:04:00.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.592+0000 7fb2e0f44700 1 Processor -- start 2024-03-24T12:04:00.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.592+0000 7fb2e0f44700 1 -- start start 2024-03-24T12:04:00.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.593+0000 7fb2e0f44700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2dc106500 0x7fb2dc1088f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:00.232 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.593+0000 7fb2e0f44700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc0796c0 con 0x7fb2dc106500 2024-03-24T12:04:00.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.593+0000 7fb2da59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2dc106500 0x7fb2dc1088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:00.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.593+0000 7fb2da59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2dc106500 0x7fb2dc1088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59350/0 (socket says 172.21.15.6:59350) 2024-03-24T12:04:00.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.593+0000 7fb2da59c700 1 -- 172.21.15.6:0/2857897471 learned_addr learned my addr 172.21.15.6:0/2857897471 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:00.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.593+0000 7fb2da59c700 1 -- 172.21.15.6:0/2857897471 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc108e30 con 0x7fb2dc106500 2024-03-24T12:04:00.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.594+0000 7fb2da59c700 1 --2- 172.21.15.6:0/2857897471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2dc106500 0x7fb2dc1088f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40098a0 tx=0x7fb2c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db2b46007f7b10d server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:00.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.594+0000 7fb2d959a700 1 -- 172.21.15.6:0/2857897471 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2c4004160 con 0x7fb2dc106500 2024-03-24T12:04:00.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.594+0000 7fb2d959a700 1 -- 172.21.15.6:0/2857897471 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fb2c40042c0 con 0x7fb2dc106500 2024-03-24T12:04:00.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.594+0000 7fb2d959a700 1 -- 172.21.15.6:0/2857897471 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2c40118a0 con 0x7fb2dc106500 2024-03-24T12:04:00.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.594+0000 7fb2e0f44700 1 -- 172.21.15.6:0/2857897471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2dc106500 msgr2=0x7fb2dc1088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:00.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.594+0000 7fb2e0f44700 1 --2- 172.21.15.6:0/2857897471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2dc106500 0x7fb2dc1088f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40098a0 tx=0x7fb2c4009bb0 comp rx=0 tx=0).stop 2024-03-24T12:04:00.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.594+0000 7fb2e0f44700 1 -- 172.21.15.6:0/2857897471 shutdown_connections 2024-03-24T12:04:00.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.594+0000 7fb2e0f44700 1 --2- 172.21.15.6:0/2857897471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2dc106500 0x7fb2dc1088f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:00.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.594+0000 7fb2e0f44700 1 -- 172.21.15.6:0/2857897471 >> 172.21.15.6:0/2857897471 conn(0x7fb2dc0ffef0 msgr2=0x7fb2dc102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:00.233 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.595+0000 7fb2e0f44700 1 -- 172.21.15.6:0/2857897471 shutdown_connections 2024-03-24T12:04:00.234 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.595+0000 7fb2e0f44700 1 -- 172.21.15.6:0/2857897471 wait complete. 2024-03-24T12:04:00.234 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.595+0000 7fb2e0f44700 1 Processor -- start 2024-03-24T12:04:00.234 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.595+0000 7fb2e0f44700 1 -- start start 2024-03-24T12:04:00.234 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.595+0000 7fb2e0f44700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2dc106500 0x7fb2dc105d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:00.234 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.595+0000 7fb2e0f44700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc106270 con 0x7fb2dc106500 2024-03-24T12:04:00.234 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.596+0000 7fb2da59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2dc106500 0x7fb2dc105d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:00.234 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.596+0000 7fb2da59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2dc106500 0x7fb2dc105d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59366/0 (socket says 172.21.15.6:59366) 2024-03-24T12:04:00.234 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.596+0000 7fb2da59c700 1 -- 172.21.15.6:0/1508659535 learned_addr learned my addr 172.21.15.6:0/1508659535 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:00.234 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.596+0000 7fb2da59c700 1 -- 172.21.15.6:0/1508659535 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c4009550 con 0x7fb2dc106500 2024-03-24T12:04:00.234 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.596+0000 7fb2da59c700 1 --2- 172.21.15.6:0/1508659535 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2dc106500 0x7fb2dc105d30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4011890 tx=0x7fb2c4011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:00.234 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.596+0000 7fb2d37fe700 1 -- 172.21.15.6:0/1508659535 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2c40036a0 con 0x7fb2dc106500 2024-03-24T12:04:00.234 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.596+0000 7fb2d37fe700 1 -- 172.21.15.6:0/1508659535 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fb2c401a430 con 0x7fb2dc106500 2024-03-24T12:04:00.234 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.596+0000 7fb2d37fe700 1 -- 172.21.15.6:0/1508659535 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2c40193c0 con 0x7fb2dc106500 2024-03-24T12:04:00.234 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.596+0000 7fb2e0f44700 1 -- 172.21.15.6:0/1508659535 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2dc1044b0 con 0x7fb2dc106500 2024-03-24T12:04:00.235 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.596+0000 7fb2e0f44700 1 -- 172.21.15.6:0/1508659535 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2dc104950 con 0x7fb2dc106500 2024-03-24T12:04:00.235 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.596+0000 7fb2d37fe700 1 -- 172.21.15.6:0/1508659535 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb2c4019980 con 0x7fb2dc106500 2024-03-24T12:04:00.235 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.597+0000 7fb2d37fe700 1 -- 172.21.15.6:0/1508659535 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb2c400b720 con 0x7fb2dc106500 2024-03-24T12:04:00.235 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.597+0000 7fb2e0f44700 1 -- 172.21.15.6:0/1508659535 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2dc0632a0 con 0x7fb2dc106500 2024-03-24T12:04:00.235 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.599+0000 7fb2d37fe700 1 -- 172.21.15.6:0/1508659535 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb2c402a060 con 0x7fb2dc106500 2024-03-24T12:04:00.235 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.673+0000 7fb2e0f44700 1 -- 172.21.15.6:0/1508659535 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb2dc109ce0 con 0x7fb2dc106500 2024-03-24T12:04:00.235 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.674+0000 7fb2d37fe700 1 -- 172.21.15.6:0/1508659535 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fb2c4019d70 con 0x7fb2dc106500 2024-03-24T12:04:00.235 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.675+0000 7fb2e0f44700 1 -- 172.21.15.6:0/1508659535 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2dc106500 msgr2=0x7fb2dc105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:00.235 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.675+0000 7fb2e0f44700 1 --2- 172.21.15.6:0/1508659535 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2dc106500 0x7fb2dc105d30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4011890 tx=0x7fb2c4011970 comp rx=0 tx=0).stop 2024-03-24T12:04:00.235 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.675+0000 7fb2e0f44700 1 -- 172.21.15.6:0/1508659535 shutdown_connections 2024-03-24T12:04:00.235 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.675+0000 7fb2e0f44700 1 --2- 172.21.15.6:0/1508659535 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2dc106500 0x7fb2dc105d30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:00.235 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.675+0000 7fb2e0f44700 1 -- 172.21.15.6:0/1508659535 >> 172.21.15.6:0/1508659535 conn(0x7fb2dc0ffef0 msgr2=0x7fb2dc1009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:00.235 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.675+0000 7fb2e0f44700 1 -- 172.21.15.6:0/1508659535 shutdown_connections 2024-03-24T12:04:00.235 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:03:59.675+0000 7fb2e0f44700 1 -- 172.21.15.6:0/1508659535 wait complete. 2024-03-24T12:04:00.236 INFO:teuthology.orchestra.run.smithi006.stdout:mgr not available, waiting (1/15)... 2024-03-24T12:04:00.461 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:00 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:00.031+0000 7f3c8ce27200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T12:04:00.461 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:00 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:00.129+0000 7f3c8ce27200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T12:04:00.461 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:00 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:00.214+0000 7f3c8ce27200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T12:04:00.787 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:00 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:00.492+0000 7f3c8ce27200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-24T12:04:00.788 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:00 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:00.604+0000 7f3c8ce27200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T12:04:00.788 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:00 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:00.691+0000 7f3c8ce27200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T12:04:00.788 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:00 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:00.788+0000 7f3c8ce27200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T12:04:01.211 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:00 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:00.966+0000 7f3c8ce27200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T12:04:02.341 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:01 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:01.962+0000 7f3c8ce27200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T12:04:02.341 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:02 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:02.095+0000 7f3c8ce27200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T12:04:02.673 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:02 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:02.445+0000 7f3c8ce27200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T12:04:02.673 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:02 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:02.542+0000 7f3c8ce27200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T12:04:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:02 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3605976226' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-24T12:04:02.961 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:02 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:02.766+0000 7f3c8ce27200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T12:04:03.262 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:02 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:02.968+0000 7f3c8ce27200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T12:04:03.262 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:03 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:03.262+0000 7f3c8ce27200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T12:04:03.303 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-24T12:04:03.303 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-03-24T12:04:03.303 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsid": "66d0ad74-e9d6-11ee-95cd-87774f69a715", 2024-03-24T12:04:03.303 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "health": { 2024-03-24T12:04:03.303 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-24T12:04:03.303 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-24T12:04:03.303 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-24T12:04:03.303 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:03.303 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-24T12:04:03.303 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-24T12:04:03.304 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 0 2024-03-24T12:04:03.304 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-24T12:04:03.304 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-24T12:04:03.304 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "a" 2024-03-24T12:04:03.304 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-24T12:04:03.304 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-03-24T12:04:03.304 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-24T12:04:03.304 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T12:04:03.304 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-24T12:04:03.304 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-24T12:04:03.304 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:03.304 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-24T12:04:03.304 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T12:04:03.304 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-24T12:04:03.304 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-24T12:04:03.305 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-24T12:04:03.312 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-24T12:04:03.312 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-24T12:04:03.313 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-24T12:04:03.313 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:03.313 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-24T12:04:03.313 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-24T12:04:03.313 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-24T12:04:03.313 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-24T12:04:03.313 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-24T12:04:03.313 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-24T12:04:03.313 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-24T12:04:03.313 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-24T12:04:03.313 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-24T12:04:03.313 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:03.313 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-24T12:04:03.313 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T12:04:03.313 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-24T12:04:03.314 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-24T12:04:03.314 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:03.314 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-24T12:04:03.314 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-24T12:04:03.314 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-24T12:04:03.314 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-24T12:04:03.314 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-24T12:04:03.314 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-24T12:04:03.314 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "restful" 2024-03-24T12:04:03.314 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-24T12:04:03.314 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T12:04:03.314 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:03.314 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-24T12:04:03.314 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T12:04:03.314 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modified": "2024-03-24T12:03:42.337658+0000", 2024-03-24T12:04:03.315 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T12:04:03.315 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:03.315 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-24T12:04:03.315 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-03-24T12:04:03.315 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.673+0000 7f7c6377f700 1 Processor -- start 2024-03-24T12:04:03.315 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.673+0000 7f7c6377f700 1 -- start start 2024-03-24T12:04:03.315 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.673+0000 7f7c6377f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c5c1042d0 0x7f7c5c1046f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:03.315 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.673+0000 7f7c6377f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c0796a0 con 0x7f7c5c1042d0 2024-03-24T12:04:03.315 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.673+0000 7f7c6151b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c5c1042d0 0x7f7c5c1046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:03.315 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.673+0000 7f7c6151b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c5c1042d0 0x7f7c5c1046f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59368/0 (socket says 172.21.15.6:59368) 2024-03-24T12:04:03.315 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.673+0000 7f7c6151b700 1 -- 172.21.15.6:0/2083379536 learned_addr learned my addr 172.21.15.6:0/2083379536 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:03.315 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.674+0000 7f7c6151b700 1 -- 172.21.15.6:0/2083379536 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5c104c30 con 0x7f7c5c1042d0 2024-03-24T12:04:03.315 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.674+0000 7f7c6151b700 1 --2- 172.21.15.6:0/2083379536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c5c1042d0 0x7f7c5c1046f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7c580098a0 tx=0x7f7c58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3999e470669e188 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:03.315 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.674+0000 7f7c53fff700 1 -- 172.21.15.6:0/2083379536 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c58004160 con 0x7f7c5c1042d0 2024-03-24T12:04:03.316 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.674+0000 7f7c53fff700 1 -- 172.21.15.6:0/2083379536 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f7c580042c0 con 0x7f7c5c1042d0 2024-03-24T12:04:03.316 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.674+0000 7f7c53fff700 1 -- 172.21.15.6:0/2083379536 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c580118a0 con 0x7f7c5c1042d0 2024-03-24T12:04:03.316 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.675+0000 7f7c6377f700 1 -- 172.21.15.6:0/2083379536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c5c1042d0 msgr2=0x7f7c5c1046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:03.316 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.675+0000 7f7c6377f700 1 --2- 172.21.15.6:0/2083379536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c5c1042d0 0x7f7c5c1046f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7c580098a0 tx=0x7f7c58009bb0 comp rx=0 tx=0).stop 2024-03-24T12:04:03.316 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.675+0000 7f7c6377f700 1 -- 172.21.15.6:0/2083379536 shutdown_connections 2024-03-24T12:04:03.316 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.675+0000 7f7c6377f700 1 --2- 172.21.15.6:0/2083379536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c5c1042d0 0x7f7c5c1046f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:03.316 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.675+0000 7f7c6377f700 1 -- 172.21.15.6:0/2083379536 >> 172.21.15.6:0/2083379536 conn(0x7f7c5c0ffe50 msgr2=0x7f7c5c1022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:03.316 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.675+0000 7f7c6377f700 1 -- 172.21.15.6:0/2083379536 shutdown_connections 2024-03-24T12:04:03.316 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.675+0000 7f7c6377f700 1 -- 172.21.15.6:0/2083379536 wait complete. 2024-03-24T12:04:03.316 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.676+0000 7f7c6377f700 1 Processor -- start 2024-03-24T12:04:03.316 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.676+0000 7f7c6377f700 1 -- start start 2024-03-24T12:04:03.316 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.676+0000 7f7c6377f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c5c1042d0 0x7f7c5c19b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:03.316 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.676+0000 7f7c6377f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c19bd60 con 0x7f7c5c1042d0 2024-03-24T12:04:03.316 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.676+0000 7f7c6151b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c5c1042d0 0x7f7c5c19b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:03.317 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.676+0000 7f7c6151b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c5c1042d0 0x7f7c5c19b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59374/0 (socket says 172.21.15.6:59374) 2024-03-24T12:04:03.317 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.676+0000 7f7c6151b700 1 -- 172.21.15.6:0/3605976226 learned_addr learned my addr 172.21.15.6:0/3605976226 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:03.317 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.677+0000 7f7c6151b700 1 -- 172.21.15.6:0/3605976226 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c58009550 con 0x7f7c5c1042d0 2024-03-24T12:04:03.317 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.677+0000 7f7c6151b700 1 --2- 172.21.15.6:0/3605976226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c5c1042d0 0x7f7c5c19b820 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7c58011890 tx=0x7f7c58011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:03.317 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.677+0000 7f7c527fc700 1 -- 172.21.15.6:0/3605976226 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c580036a0 con 0x7f7c5c1042d0 2024-03-24T12:04:03.317 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.677+0000 7f7c6377f700 1 -- 172.21.15.6:0/3605976226 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5c19bf60 con 0x7f7c5c1042d0 2024-03-24T12:04:03.317 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.677+0000 7f7c6377f700 1 -- 172.21.15.6:0/3605976226 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c5c19c400 con 0x7f7c5c1042d0 2024-03-24T12:04:03.317 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.677+0000 7f7c527fc700 1 -- 172.21.15.6:0/3605976226 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f7c5801a430 con 0x7f7c5c1042d0 2024-03-24T12:04:03.317 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.677+0000 7f7c527fc700 1 -- 172.21.15.6:0/3605976226 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c580193c0 con 0x7f7c5c1042d0 2024-03-24T12:04:03.317 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.678+0000 7f7c527fc700 1 -- 172.21.15.6:0/3605976226 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7c58019980 con 0x7f7c5c1042d0 2024-03-24T12:04:03.317 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.678+0000 7f7c527fc700 1 -- 172.21.15.6:0/3605976226 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7c5800b720 con 0x7f7c5c1042d0 2024-03-24T12:04:03.317 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.679+0000 7f7c6377f700 1 -- 172.21.15.6:0/3605976226 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c40005150 con 0x7f7c5c1042d0 2024-03-24T12:04:03.317 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.683+0000 7f7c527fc700 1 -- 172.21.15.6:0/3605976226 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7c5802a060 con 0x7f7c5c1042d0 2024-03-24T12:04:03.317 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.775+0000 7f7c6377f700 1 -- 172.21.15.6:0/3605976226 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f7c400056d0 con 0x7f7c5c1042d0 2024-03-24T12:04:03.317 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.776+0000 7f7c527fc700 1 -- 172.21.15.6:0/3605976226 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f7c58019d70 con 0x7f7c5c1042d0 2024-03-24T12:04:03.318 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.777+0000 7f7c6377f700 1 -- 172.21.15.6:0/3605976226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c5c1042d0 msgr2=0x7f7c5c19b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:03.318 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.777+0000 7f7c6377f700 1 --2- 172.21.15.6:0/3605976226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c5c1042d0 0x7f7c5c19b820 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7c58011890 tx=0x7f7c58011970 comp rx=0 tx=0).stop 2024-03-24T12:04:03.318 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.777+0000 7f7c6377f700 1 -- 172.21.15.6:0/3605976226 shutdown_connections 2024-03-24T12:04:03.318 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.777+0000 7f7c6377f700 1 --2- 172.21.15.6:0/3605976226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c5c1042d0 0x7f7c5c19b820 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:03.318 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.777+0000 7f7c6377f700 1 -- 172.21.15.6:0/3605976226 >> 172.21.15.6:0/3605976226 conn(0x7f7c5c0ffe50 msgr2=0x7f7c5c192690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:03.318 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.777+0000 7f7c6377f700 1 -- 172.21.15.6:0/3605976226 shutdown_connections 2024-03-24T12:04:03.318 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:02.777+0000 7f7c6377f700 1 -- 172.21.15.6:0/3605976226 wait complete. 2024-03-24T12:04:03.318 INFO:teuthology.orchestra.run.smithi006.stdout:mgr not available, waiting (2/15)... 2024-03-24T12:04:03.961 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:03 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:03.526+0000 7f3c8ce27200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T12:04:03.961 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:03 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:03.766+0000 7f3c8ce27200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T12:04:04.961 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:04 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:04.585+0000 7f3c8ce27200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T12:04:05.430 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:05.167+0000 7f3c8ce27200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T12:04:05.431 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:05.267+0000 7f3c8ce27200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T12:04:05.681 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:05 smithi006 ceph-mon[30108]: Activating manager daemon y 2024-03-24T12:04:05.681 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:05 smithi006 ceph-mon[30108]: mgrmap e2: y(active, starting, since 0.00283874s) 2024-03-24T12:04:05.681 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:05 smithi006 ceph-mon[30108]: from='mgr.14100 172.21.15.6:0/2250741720' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T12:04:05.681 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:05 smithi006 ceph-mon[30108]: from='mgr.14100 172.21.15.6:0/2250741720' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T12:04:05.681 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:05 smithi006 ceph-mon[30108]: from='mgr.14100 172.21.15.6:0/2250741720' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T12:04:05.682 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:05 smithi006 ceph-mon[30108]: from='mgr.14100 172.21.15.6:0/2250741720' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T12:04:05.682 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:05 smithi006 ceph-mon[30108]: from='mgr.14100 172.21.15.6:0/2250741720' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T12:04:05.682 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:05 smithi006 ceph-mon[30108]: Manager daemon y is now available 2024-03-24T12:04:05.682 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:05 smithi006 ceph-mon[30108]: from='mgr.14100 172.21.15.6:0/2250741720' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T12:04:05.682 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:05 smithi006 ceph-mon[30108]: from='mgr.14100 172.21.15.6:0/2250741720' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T12:04:05.682 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:05 smithi006 ceph-mon[30108]: from='mgr.14100 172.21.15.6:0/2250741720' entity='mgr.y' 2024-03-24T12:04:05.682 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:05 smithi006 ceph-mon[30108]: from='mgr.14100 172.21.15.6:0/2250741720' entity='mgr.y' 2024-03-24T12:04:05.682 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:05 smithi006 ceph-mon[30108]: from='mgr.14100 172.21.15.6:0/2250741720' entity='mgr.y' 2024-03-24T12:04:05.682 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:05.490+0000 7f3c8ce27200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T12:04:06.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-24T12:04:06.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-03-24T12:04:06.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsid": "66d0ad74-e9d6-11ee-95cd-87774f69a715", 2024-03-24T12:04:06.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "health": { 2024-03-24T12:04:06.450 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-24T12:04:06.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-24T12:04:06.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-24T12:04:06.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:06.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-24T12:04:06.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-24T12:04:06.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 0 2024-03-24T12:04:06.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-24T12:04:06.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-24T12:04:06.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "a" 2024-03-24T12:04:06.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-24T12:04:06.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-03-24T12:04:06.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-24T12:04:06.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T12:04:06.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-24T12:04:06.451 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-24T12:04:06.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:06.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-24T12:04:06.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T12:04:06.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-24T12:04:06.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-24T12:04:06.452 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-24T12:04:06.459 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-24T12:04:06.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-24T12:04:06.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-24T12:04:06.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:06.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-24T12:04:06.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-24T12:04:06.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-24T12:04:06.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-24T12:04:06.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-24T12:04:06.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-24T12:04:06.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-24T12:04:06.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-24T12:04:06.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-24T12:04:06.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:06.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-24T12:04:06.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T12:04:06.460 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-24T12:04:06.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-24T12:04:06.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:06.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-24T12:04:06.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-24T12:04:06.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-24T12:04:06.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-24T12:04:06.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-24T12:04:06.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-24T12:04:06.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "restful" 2024-03-24T12:04:06.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-24T12:04:06.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T12:04:06.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:06.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-24T12:04:06.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T12:04:06.461 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modified": "2024-03-24T12:03:42.337658+0000", 2024-03-24T12:04:06.462 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T12:04:06.462 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:06.462 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-24T12:04:06.462 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-03-24T12:04:06.462 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.776+0000 7f2218810700 1 Processor -- start 2024-03-24T12:04:06.462 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.776+0000 7f2218810700 1 -- start start 2024-03-24T12:04:06.462 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.776+0000 7f2218810700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2214106550 0x7f2214108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:06.462 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.776+0000 7f2218810700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2214078db0 con 0x7f2214106550 2024-03-24T12:04:06.462 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.776+0000 7f221259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2214106550 0x7f2214108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:06.462 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.776+0000 7f221259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2214106550 0x7f2214108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35880/0 (socket says 172.21.15.6:35880) 2024-03-24T12:04:06.462 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.776+0000 7f221259c700 1 -- 172.21.15.6:0/2964643605 learned_addr learned my addr 172.21.15.6:0/2964643605 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:06.462 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.776+0000 7f221259c700 1 -- 172.21.15.6:0/2964643605 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2214078ef0 con 0x7f2214106550 2024-03-24T12:04:06.462 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.777+0000 7f221259c700 1 --2- 172.21.15.6:0/2964643605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2214106550 0x7f2214108940 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f22040098a0 tx=0x7f2204009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6158ff46ae92643 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:06.463 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.777+0000 7f221159a700 1 -- 172.21.15.6:0/2964643605 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2204004160 con 0x7f2214106550 2024-03-24T12:04:06.463 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.777+0000 7f221159a700 1 -- 172.21.15.6:0/2964643605 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f22040042c0 con 0x7f2214106550 2024-03-24T12:04:06.463 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.777+0000 7f221159a700 1 -- 172.21.15.6:0/2964643605 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22040118a0 con 0x7f2214106550 2024-03-24T12:04:06.463 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.777+0000 7f2218810700 1 -- 172.21.15.6:0/2964643605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2214106550 msgr2=0x7f2214108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:06.463 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.777+0000 7f2218810700 1 --2- 172.21.15.6:0/2964643605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2214106550 0x7f2214108940 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f22040098a0 tx=0x7f2204009bb0 comp rx=0 tx=0).stop 2024-03-24T12:04:06.463 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.777+0000 7f2218810700 1 -- 172.21.15.6:0/2964643605 shutdown_connections 2024-03-24T12:04:06.463 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.777+0000 7f2218810700 1 --2- 172.21.15.6:0/2964643605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2214106550 0x7f2214108940 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:06.463 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.777+0000 7f2218810700 1 -- 172.21.15.6:0/2964643605 >> 172.21.15.6:0/2964643605 conn(0x7f22140ffec0 msgr2=0x7f22141022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:06.463 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.778+0000 7f2218810700 1 -- 172.21.15.6:0/2964643605 shutdown_connections 2024-03-24T12:04:06.463 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.778+0000 7f2218810700 1 -- 172.21.15.6:0/2964643605 wait complete. 2024-03-24T12:04:06.463 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.778+0000 7f2218810700 1 Processor -- start 2024-03-24T12:04:06.463 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.778+0000 7f2218810700 1 -- start start 2024-03-24T12:04:06.463 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.778+0000 7f2218810700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2214106550 0x7f2214199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:06.463 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.778+0000 7f2218810700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2214199bc0 con 0x7f2214106550 2024-03-24T12:04:06.464 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.778+0000 7f221259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2214106550 0x7f2214199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:06.464 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.778+0000 7f221259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2214106550 0x7f2214199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35884/0 (socket says 172.21.15.6:35884) 2024-03-24T12:04:06.464 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.778+0000 7f221259c700 1 -- 172.21.15.6:0/2228682750 learned_addr learned my addr 172.21.15.6:0/2228682750 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:06.464 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.779+0000 7f221259c700 1 -- 172.21.15.6:0/2228682750 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2204009550 con 0x7f2214106550 2024-03-24T12:04:06.464 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.779+0000 7f221259c700 1 --2- 172.21.15.6:0/2228682750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2214106550 0x7f2214199680 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f22040046c0 tx=0x7f2204011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:06.464 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.779+0000 7f22037fe700 1 -- 172.21.15.6:0/2228682750 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22040036a0 con 0x7f2214106550 2024-03-24T12:04:06.464 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.779+0000 7f22037fe700 1 -- 172.21.15.6:0/2228682750 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f220401a430 con 0x7f2214106550 2024-03-24T12:04:06.464 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.779+0000 7f2218810700 1 -- 172.21.15.6:0/2228682750 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2214199dc0 con 0x7f2214106550 2024-03-24T12:04:06.464 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.779+0000 7f22037fe700 1 -- 172.21.15.6:0/2228682750 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22040193c0 con 0x7f2214106550 2024-03-24T12:04:06.464 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.779+0000 7f2218810700 1 -- 172.21.15.6:0/2228682750 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f221419a260 con 0x7f2214106550 2024-03-24T12:04:06.464 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.780+0000 7f22037fe700 1 -- 172.21.15.6:0/2228682750 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 2) v1 ==== 45525+0+0 (secure 0 0 0) 0x7f22040195f0 con 0x7f2214106550 2024-03-24T12:04:06.464 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.780+0000 7f2218810700 1 -- 172.21.15.6:0/2228682750 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2214109a80 con 0x7f2214106550 2024-03-24T12:04:06.464 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.780+0000 7f22037fe700 1 -- 172.21.15.6:0/2228682750 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f220404c870 con 0x7f2214106550 2024-03-24T12:04:06.465 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.782+0000 7f22037fe700 1 -- 172.21.15.6:0/2228682750 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f2204018350 con 0x7f2214106550 2024-03-24T12:04:06.465 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.855+0000 7f2218810700 1 -- 172.21.15.6:0/2228682750 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f221419ccc0 con 0x7f2214106550 2024-03-24T12:04:06.465 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.855+0000 7f22037fe700 1 -- 172.21.15.6:0/2228682750 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f2204040e60 con 0x7f2214106550 2024-03-24T12:04:06.465 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.856+0000 7f2218810700 1 -- 172.21.15.6:0/2228682750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2214106550 msgr2=0x7f2214199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:06.465 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.856+0000 7f2218810700 1 --2- 172.21.15.6:0/2228682750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2214106550 0x7f2214199680 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f22040046c0 tx=0x7f2204011eb0 comp rx=0 tx=0).stop 2024-03-24T12:04:06.465 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.856+0000 7f2218810700 1 -- 172.21.15.6:0/2228682750 shutdown_connections 2024-03-24T12:04:06.465 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.856+0000 7f2218810700 1 --2- 172.21.15.6:0/2228682750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2214106550 0x7f2214199680 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:06.465 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.856+0000 7f2218810700 1 -- 172.21.15.6:0/2228682750 >> 172.21.15.6:0/2228682750 conn(0x7f22140ffec0 msgr2=0x7f2214100a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:06.465 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.856+0000 7f2218810700 1 -- 172.21.15.6:0/2228682750 shutdown_connections 2024-03-24T12:04:06.465 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:05.856+0000 7f2218810700 1 -- 172.21.15.6:0/2228682750 wait complete. 2024-03-24T12:04:06.465 INFO:teuthology.orchestra.run.smithi006.stdout:mgr not available, waiting (3/15)... 2024-03-24T12:04:06.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:06 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2228682750' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-24T12:04:06.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:06 smithi006 ceph-mon[30108]: mgrmap e3: y(active, since 1.0064s) 2024-03-24T12:04:08.856 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:08 smithi006 ceph-mon[30108]: mgrmap e4: y(active, since 2s) 2024-03-24T12:04:09.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:09 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2659196086' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-24T12:04:09.723 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-24T12:04:09.723 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-03-24T12:04:09.723 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsid": "66d0ad74-e9d6-11ee-95cd-87774f69a715", 2024-03-24T12:04:09.723 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "health": { 2024-03-24T12:04:09.723 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-24T12:04:09.724 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-24T12:04:09.724 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-24T12:04:09.724 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:09.724 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-24T12:04:09.724 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-24T12:04:09.724 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 0 2024-03-24T12:04:09.724 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-24T12:04:09.724 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-24T12:04:09.724 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "a" 2024-03-24T12:04:09.724 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-24T12:04:09.724 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-03-24T12:04:09.724 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-24T12:04:09.724 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T12:04:09.724 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-24T12:04:09.724 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-24T12:04:09.724 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:09.724 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-24T12:04:09.725 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T12:04:09.725 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-24T12:04:09.725 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-24T12:04:09.733 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-24T12:04:09.733 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-24T12:04:09.733 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-24T12:04:09.733 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-24T12:04:09.733 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:09.733 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-24T12:04:09.733 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-24T12:04:09.733 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-24T12:04:09.733 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-24T12:04:09.733 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-24T12:04:09.733 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-24T12:04:09.734 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-24T12:04:09.734 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-24T12:04:09.734 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-24T12:04:09.734 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:09.734 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-24T12:04:09.734 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T12:04:09.734 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-24T12:04:09.734 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-24T12:04:09.734 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:09.734 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-24T12:04:09.734 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-24T12:04:09.734 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-24T12:04:09.734 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-24T12:04:09.734 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-24T12:04:09.734 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-24T12:04:09.735 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "restful" 2024-03-24T12:04:09.735 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ], 2024-03-24T12:04:09.735 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T12:04:09.735 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:09.735 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-24T12:04:09.735 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-24T12:04:09.735 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "modified": "2024-03-24T12:03:42.337658+0000", 2024-03-24T12:04:09.735 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-24T12:04:09.735 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout }, 2024-03-24T12:04:09.735 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-24T12:04:09.735 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-03-24T12:04:09.735 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.886+0000 7f1e75a40700 1 Processor -- start 2024-03-24T12:04:09.735 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.886+0000 7f1e75a40700 1 -- start start 2024-03-24T12:04:09.735 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.886+0000 7f1e75a40700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e70079b70 0x7f1e70079f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:09.736 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.886+0000 7f1e75a40700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7007b8a0 con 0x7f1e70079b70 2024-03-24T12:04:09.736 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.886+0000 7f1e6effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e70079b70 0x7f1e70079f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:09.736 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.886+0000 7f1e6effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e70079b70 0x7f1e70079f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35894/0 (socket says 172.21.15.6:35894) 2024-03-24T12:04:09.736 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.886+0000 7f1e6effd700 1 -- 172.21.15.6:0/743146385 learned_addr learned my addr 172.21.15.6:0/743146385 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:09.736 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.886+0000 7f1e6effd700 1 -- 172.21.15.6:0/743146385 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7007a4d0 con 0x7f1e70079b70 2024-03-24T12:04:09.736 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.886+0000 7f1e6effd700 1 --2- 172.21.15.6:0/743146385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e70079b70 0x7f1e70079f90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1e600098a0 tx=0x7f1e60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846f880edadf1314 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:09.736 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.887+0000 7f1e6dffb700 1 -- 172.21.15.6:0/743146385 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e60004160 con 0x7f1e70079b70 2024-03-24T12:04:09.736 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.887+0000 7f1e6dffb700 1 -- 172.21.15.6:0/743146385 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f1e600042c0 con 0x7f1e70079b70 2024-03-24T12:04:09.736 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.887+0000 7f1e6dffb700 1 -- 172.21.15.6:0/743146385 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e600118a0 con 0x7f1e70079b70 2024-03-24T12:04:09.736 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.887+0000 7f1e75a40700 1 -- 172.21.15.6:0/743146385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e70079b70 msgr2=0x7f1e70079f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:09.736 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.887+0000 7f1e75a40700 1 --2- 172.21.15.6:0/743146385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e70079b70 0x7f1e70079f90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1e600098a0 tx=0x7f1e60009bb0 comp rx=0 tx=0).stop 2024-03-24T12:04:09.736 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.887+0000 7f1e75a40700 1 -- 172.21.15.6:0/743146385 shutdown_connections 2024-03-24T12:04:09.736 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.887+0000 7f1e75a40700 1 --2- 172.21.15.6:0/743146385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e70079b70 0x7f1e70079f90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:09.736 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.887+0000 7f1e75a40700 1 -- 172.21.15.6:0/743146385 >> 172.21.15.6:0/743146385 conn(0x7f1e70101030 msgr2=0x7f1e70103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:09.736 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.887+0000 7f1e75a40700 1 -- 172.21.15.6:0/743146385 shutdown_connections 2024-03-24T12:04:09.737 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.887+0000 7f1e75a40700 1 -- 172.21.15.6:0/743146385 wait complete. 2024-03-24T12:04:09.737 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.888+0000 7f1e75a40700 1 Processor -- start 2024-03-24T12:04:09.737 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.888+0000 7f1e75a40700 1 -- start start 2024-03-24T12:04:09.737 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.888+0000 7f1e75a40700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e70079b70 0x7f1e7019b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:09.737 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.888+0000 7f1e75a40700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7019beb0 con 0x7f1e70079b70 2024-03-24T12:04:09.737 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.888+0000 7f1e6effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e70079b70 0x7f1e7019b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:09.737 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.888+0000 7f1e6effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e70079b70 0x7f1e7019b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35898/0 (socket says 172.21.15.6:35898) 2024-03-24T12:04:09.737 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.888+0000 7f1e6effd700 1 -- 172.21.15.6:0/2659196086 learned_addr learned my addr 172.21.15.6:0/2659196086 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:09.737 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.889+0000 7f1e6effd700 1 -- 172.21.15.6:0/2659196086 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e60009550 con 0x7f1e70079b70 2024-03-24T12:04:09.737 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.889+0000 7f1e6effd700 1 --2- 172.21.15.6:0/2659196086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e70079b70 0x7f1e7019b970 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1e60011890 tx=0x7f1e60011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:09.737 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.889+0000 7f1e74a3e700 1 -- 172.21.15.6:0/2659196086 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e600036a0 con 0x7f1e70079b70 2024-03-24T12:04:09.737 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.889+0000 7f1e74a3e700 1 -- 172.21.15.6:0/2659196086 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f1e6001a430 con 0x7f1e70079b70 2024-03-24T12:04:09.737 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.889+0000 7f1e75a40700 1 -- 172.21.15.6:0/2659196086 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7019c0b0 con 0x7f1e70079b70 2024-03-24T12:04:09.737 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.889+0000 7f1e74a3e700 1 -- 172.21.15.6:0/2659196086 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e600193c0 con 0x7f1e70079b70 2024-03-24T12:04:09.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.889+0000 7f1e75a40700 1 -- 172.21.15.6:0/2659196086 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7019c550 con 0x7f1e70079b70 2024-03-24T12:04:09.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.890+0000 7f1e74a3e700 1 -- 172.21.15.6:0/2659196086 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) v1 ==== 45758+0+0 (secure 0 0 0) 0x7f1e600195f0 con 0x7f1e70079b70 2024-03-24T12:04:09.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.890+0000 7f1e74a3e700 1 --2- 172.21.15.6:0/2659196086 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f1e58038cb0 0x7f1e5803b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:09.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.890+0000 7f1e74a3e700 1 -- 172.21.15.6:0/2659196086 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1e6004cba0 con 0x7f1e70079b70 2024-03-24T12:04:09.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.890+0000 7f1e75a40700 1 -- 172.21.15.6:0/2659196086 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7010bdc0 con 0x7f1e70079b70 2024-03-24T12:04:09.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.890+0000 7f1e6e7fc700 1 --2- 172.21.15.6:0/2659196086 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f1e58038cb0 0x7f1e5803b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:09.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.890+0000 7f1e6e7fc700 1 --2- 172.21.15.6:0/2659196086 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f1e58038cb0 0x7f1e5803b170 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400ad30 tx=0x7f1e640093f0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:09.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:08.895+0000 7f1e74a3e700 1 -- 172.21.15.6:0/2659196086 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1e60018940 con 0x7f1e70079b70 2024-03-24T12:04:09.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:09.181+0000 7f1e75a40700 1 -- 172.21.15.6:0/2659196086 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f1e7019ede0 con 0x7f1e70079b70 2024-03-24T12:04:09.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:09.182+0000 7f1e74a3e700 1 -- 172.21.15.6:0/2659196086 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f1e6002d350 con 0x7f1e70079b70 2024-03-24T12:04:09.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:09.184+0000 7f1e75a40700 1 -- 172.21.15.6:0/2659196086 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f1e58038cb0 msgr2=0x7f1e5803b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:09.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:09.184+0000 7f1e75a40700 1 --2- 172.21.15.6:0/2659196086 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f1e58038cb0 0x7f1e5803b170 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400ad30 tx=0x7f1e640093f0 comp rx=0 tx=0).stop 2024-03-24T12:04:09.738 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:09.184+0000 7f1e75a40700 1 -- 172.21.15.6:0/2659196086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e70079b70 msgr2=0x7f1e7019b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:09.739 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:09.184+0000 7f1e75a40700 1 --2- 172.21.15.6:0/2659196086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e70079b70 0x7f1e7019b970 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1e60011890 tx=0x7f1e60011970 comp rx=0 tx=0).stop 2024-03-24T12:04:09.739 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:09.184+0000 7f1e75a40700 1 -- 172.21.15.6:0/2659196086 shutdown_connections 2024-03-24T12:04:09.739 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:09.184+0000 7f1e75a40700 1 --2- 172.21.15.6:0/2659196086 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f1e58038cb0 0x7f1e5803b170 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:09.739 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:09.184+0000 7f1e75a40700 1 --2- 172.21.15.6:0/2659196086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e70079b70 0x7f1e7019b970 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:09.739 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:09.184+0000 7f1e75a40700 1 -- 172.21.15.6:0/2659196086 >> 172.21.15.6:0/2659196086 conn(0x7f1e70101030 msgr2=0x7f1e70101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:09.739 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:09.184+0000 7f1e75a40700 1 -- 172.21.15.6:0/2659196086 shutdown_connections 2024-03-24T12:04:09.739 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:09.184+0000 7f1e75a40700 1 -- 172.21.15.6:0/2659196086 wait complete. 2024-03-24T12:04:09.739 INFO:teuthology.orchestra.run.smithi006.stdout:mgr is available 2024-03-24T12:04:10.699 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3792844318' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-24T12:04:11.009 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-24T12:04:11.009 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [global] 2024-03-24T12:04:11.009 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout fsid = 66d0ad74-e9d6-11ee-95cd-87774f69a715 2024-03-24T12:04:11.009 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-24T12:04:11.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-24T12:04:11.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-24T12:04:11.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-24T12:04:11.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-24T12:04:11.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-24T12:04:11.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-24T12:04:11.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 2024-03-24T12:04:11.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout [osd] 2024-03-24T12:04:11.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-24T12:04:11.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-24T12:04:11.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.190+0000 7f0fca71d700 1 Processor -- start 2024-03-24T12:04:11.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.190+0000 7f0fca71d700 1 -- start start 2024-03-24T12:04:11.010 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.191+0000 7f0fca71d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fc4079b70 0x7f0fc4079f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:11.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.191+0000 7f0fca71d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc407b8a0 con 0x7f0fc4079b70 2024-03-24T12:04:11.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.191+0000 7f0fc3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fc4079b70 0x7f0fc4079f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:11.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.191+0000 7f0fc3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fc4079b70 0x7f0fc4079f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35906/0 (socket says 172.21.15.6:35906) 2024-03-24T12:04:11.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.191+0000 7f0fc3fff700 1 -- 172.21.15.6:0/3462348617 learned_addr learned my addr 172.21.15.6:0/3462348617 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:11.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.191+0000 7f0fc3fff700 1 -- 172.21.15.6:0/3462348617 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc407a4d0 con 0x7f0fc4079b70 2024-03-24T12:04:11.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.191+0000 7f0fc3fff700 1 --2- 172.21.15.6:0/3462348617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fc4079b70 0x7f0fc4079f90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40098a0 tx=0x7f0fb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4deb1805d3415a42 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:11.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.191+0000 7f0fc2ffd700 1 -- 172.21.15.6:0/3462348617 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0fb4004160 con 0x7f0fc4079b70 2024-03-24T12:04:11.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.191+0000 7f0fc2ffd700 1 -- 172.21.15.6:0/3462348617 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f0fb40042c0 con 0x7f0fc4079b70 2024-03-24T12:04:11.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.191+0000 7f0fc2ffd700 1 -- 172.21.15.6:0/3462348617 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0fb40118a0 con 0x7f0fc4079b70 2024-03-24T12:04:11.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.192+0000 7f0fca71d700 1 -- 172.21.15.6:0/3462348617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fc4079b70 msgr2=0x7f0fc4079f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:11.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.192+0000 7f0fca71d700 1 --2- 172.21.15.6:0/3462348617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fc4079b70 0x7f0fc4079f90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40098a0 tx=0x7f0fb4009bb0 comp rx=0 tx=0).stop 2024-03-24T12:04:11.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.192+0000 7f0fca71d700 1 -- 172.21.15.6:0/3462348617 shutdown_connections 2024-03-24T12:04:11.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.192+0000 7f0fca71d700 1 --2- 172.21.15.6:0/3462348617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fc4079b70 0x7f0fc4079f90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:11.011 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.192+0000 7f0fca71d700 1 -- 172.21.15.6:0/3462348617 >> 172.21.15.6:0/3462348617 conn(0x7f0fc4101030 msgr2=0x7f0fc4103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.192+0000 7f0fca71d700 1 -- 172.21.15.6:0/3462348617 shutdown_connections 2024-03-24T12:04:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.192+0000 7f0fca71d700 1 -- 172.21.15.6:0/3462348617 wait complete. 2024-03-24T12:04:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.193+0000 7f0fca71d700 1 Processor -- start 2024-03-24T12:04:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.193+0000 7f0fca71d700 1 -- start start 2024-03-24T12:04:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.193+0000 7f0fca71d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fc4079b70 0x7f0fc419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.193+0000 7f0fca71d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc419be40 con 0x7f0fc4079b70 2024-03-24T12:04:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.193+0000 7f0fc3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fc4079b70 0x7f0fc419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.193+0000 7f0fc3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fc4079b70 0x7f0fc419b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35916/0 (socket says 172.21.15.6:35916) 2024-03-24T12:04:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.193+0000 7f0fc3fff700 1 -- 172.21.15.6:0/3792844318 learned_addr learned my addr 172.21.15.6:0/3792844318 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.193+0000 7f0fc3fff700 1 -- 172.21.15.6:0/3792844318 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb4009550 con 0x7f0fc4079b70 2024-03-24T12:04:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.193+0000 7f0fc3fff700 1 --2- 172.21.15.6:0/3792844318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fc4079b70 0x7f0fc419b900 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4011890 tx=0x7f0fb4011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.194+0000 7f0fc17fa700 1 -- 172.21.15.6:0/3792844318 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0fb40036a0 con 0x7f0fc4079b70 2024-03-24T12:04:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.194+0000 7f0fc17fa700 1 -- 172.21.15.6:0/3792844318 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f0fb401a430 con 0x7f0fc4079b70 2024-03-24T12:04:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.194+0000 7f0fca71d700 1 -- 172.21.15.6:0/3792844318 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc419c040 con 0x7f0fc4079b70 2024-03-24T12:04:11.012 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.194+0000 7f0fca71d700 1 -- 172.21.15.6:0/3792844318 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc419c4e0 con 0x7f0fc4079b70 2024-03-24T12:04:11.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.194+0000 7f0fc17fa700 1 -- 172.21.15.6:0/3792844318 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0fb40193c0 con 0x7f0fc4079b70 2024-03-24T12:04:11.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.194+0000 7f0fc17fa700 1 -- 172.21.15.6:0/3792844318 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) v1 ==== 45758+0+0 (secure 0 0 0) 0x7f0fb4019640 con 0x7f0fc4079b70 2024-03-24T12:04:11.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.195+0000 7f0fc17fa700 1 --2- 172.21.15.6:0/3792844318 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f0fb0038c60 0x7f0fb003b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:11.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.195+0000 7f0fca71d700 1 -- 172.21.15.6:0/3792844318 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc410bdc0 con 0x7f0fc4079b70 2024-03-24T12:04:11.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.195+0000 7f0fc17fa700 1 -- 172.21.15.6:0/3792844318 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0fb404cc10 con 0x7f0fc4079b70 2024-03-24T12:04:11.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.195+0000 7f0fc37fe700 1 --2- 172.21.15.6:0/3792844318 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f0fb0038c60 0x7f0fb003b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:11.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.195+0000 7f0fc37fe700 1 --2- 172.21.15.6:0/3792844318 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f0fb0038c60 0x7f0fb003b120 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00ad30 tx=0x7f0fac0093f0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:11.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.200+0000 7f0fc17fa700 1 -- 172.21.15.6:0/3792844318 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0fb4018940 con 0x7f0fc4079b70 2024-03-24T12:04:11.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.402+0000 7f0fca71d700 1 -- 172.21.15.6:0/3792844318 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f0fc410bfd0 con 0x7f0fc4079b70 2024-03-24T12:04:11.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.403+0000 7f0fc17fa700 1 -- 172.21.15.6:0/3792844318 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+282 (secure 0 0 0) 0x7f0fb4018350 con 0x7f0fc4079b70 2024-03-24T12:04:11.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.405+0000 7f0fca71d700 1 -- 172.21.15.6:0/3792844318 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f0fb0038c60 msgr2=0x7f0fb003b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:11.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.405+0000 7f0fca71d700 1 --2- 172.21.15.6:0/3792844318 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f0fb0038c60 0x7f0fb003b120 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00ad30 tx=0x7f0fac0093f0 comp rx=0 tx=0).stop 2024-03-24T12:04:11.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.405+0000 7f0fca71d700 1 -- 172.21.15.6:0/3792844318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fc4079b70 msgr2=0x7f0fc419b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:11.013 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.405+0000 7f0fca71d700 1 --2- 172.21.15.6:0/3792844318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fc4079b70 0x7f0fc419b900 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4011890 tx=0x7f0fb4011970 comp rx=0 tx=0).stop 2024-03-24T12:04:11.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.405+0000 7f0fca71d700 1 -- 172.21.15.6:0/3792844318 shutdown_connections 2024-03-24T12:04:11.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.405+0000 7f0fca71d700 1 --2- 172.21.15.6:0/3792844318 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f0fb0038c60 0x7f0fb003b120 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:11.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.405+0000 7f0fca71d700 1 --2- 172.21.15.6:0/3792844318 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fc4079b70 0x7f0fc419b900 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:11.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.405+0000 7f0fca71d700 1 -- 172.21.15.6:0/3792844318 >> 172.21.15.6:0/3792844318 conn(0x7f0fc4101030 msgr2=0x7f0fc4101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:11.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.405+0000 7f0fca71d700 1 -- 172.21.15.6:0/3792844318 shutdown_connections 2024-03-24T12:04:11.014 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:10.405+0000 7f0fca71d700 1 -- 172.21.15.6:0/3792844318 wait complete. 2024-03-24T12:04:11.014 INFO:teuthology.orchestra.run.smithi006.stdout:Enabling cephadm module... 2024-03-24T12:04:12.090 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1431833931' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-24T12:04:12.090 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:11 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ignoring --setuser ceph since I am not root 2024-03-24T12:04:12.090 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:11 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ignoring --setgroup ceph since I am not root 2024-03-24T12:04:12.357 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.469+0000 7f5b4b6ce700 1 Processor -- start 2024-03-24T12:04:12.357 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.469+0000 7f5b4b6ce700 1 -- start start 2024-03-24T12:04:12.357 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.469+0000 7f5b4b6ce700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b44106470 0x7f5b44108860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:12.357 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.469+0000 7f5b4b6ce700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b44078e90 con 0x7f5b44106470 2024-03-24T12:04:12.357 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.469+0000 7f5b4946a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b44106470 0x7f5b44108860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:12.357 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.470+0000 7f5b4946a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b44106470 0x7f5b44108860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35930/0 (socket says 172.21.15.6:35930) 2024-03-24T12:04:12.357 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.470+0000 7f5b4946a700 1 -- 172.21.15.6:0/3621861339 learned_addr learned my addr 172.21.15.6:0/3621861339 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:12.357 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.470+0000 7f5b4946a700 1 -- 172.21.15.6:0/3621861339 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b44108da0 con 0x7f5b44106470 2024-03-24T12:04:12.357 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.470+0000 7f5b4946a700 1 --2- 172.21.15.6:0/3621861339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b44106470 0x7f5b44108860 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009e80 tx=0x7f5b3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dca14c4850cfbb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:12.357 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.470+0000 7f5b43fff700 1 -- 172.21.15.6:0/3621861339 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b3c00e070 con 0x7f5b44106470 2024-03-24T12:04:12.357 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.470+0000 7f5b43fff700 1 -- 172.21.15.6:0/3621861339 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f5b3c003e80 con 0x7f5b44106470 2024-03-24T12:04:12.357 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.470+0000 7f5b43fff700 1 -- 172.21.15.6:0/3621861339 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b3c004370 con 0x7f5b44106470 2024-03-24T12:04:12.357 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.471+0000 7f5b4b6ce700 1 -- 172.21.15.6:0/3621861339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b44106470 msgr2=0x7f5b44108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:12.358 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.471+0000 7f5b4b6ce700 1 --2- 172.21.15.6:0/3621861339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b44106470 0x7f5b44108860 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009e80 tx=0x7f5b3c00b330 comp rx=0 tx=0).stop 2024-03-24T12:04:12.358 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.471+0000 7f5b4b6ce700 1 -- 172.21.15.6:0/3621861339 shutdown_connections 2024-03-24T12:04:12.358 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.471+0000 7f5b4b6ce700 1 --2- 172.21.15.6:0/3621861339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b44106470 0x7f5b44108860 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:12.358 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.471+0000 7f5b4b6ce700 1 -- 172.21.15.6:0/3621861339 >> 172.21.15.6:0/3621861339 conn(0x7f5b440ffec0 msgr2=0x7f5b441022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:12.358 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.471+0000 7f5b4b6ce700 1 -- 172.21.15.6:0/3621861339 shutdown_connections 2024-03-24T12:04:12.358 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.471+0000 7f5b4b6ce700 1 -- 172.21.15.6:0/3621861339 wait complete. 2024-03-24T12:04:12.358 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.471+0000 7f5b4b6ce700 1 Processor -- start 2024-03-24T12:04:12.358 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.471+0000 7f5b4b6ce700 1 -- start start 2024-03-24T12:04:12.358 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.472+0000 7f5b4b6ce700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b44106470 0x7f5b44105cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:12.358 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.472+0000 7f5b4b6ce700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b44106230 con 0x7f5b44106470 2024-03-24T12:04:12.358 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.472+0000 7f5b4946a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b44106470 0x7f5b44105cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:12.358 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.472+0000 7f5b4946a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b44106470 0x7f5b44105cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35946/0 (socket says 172.21.15.6:35946) 2024-03-24T12:04:12.358 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.472+0000 7f5b4946a700 1 -- 172.21.15.6:0/1431833931 learned_addr learned my addr 172.21.15.6:0/1431833931 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:12.359 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.472+0000 7f5b4946a700 1 -- 172.21.15.6:0/1431833931 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3c009b30 con 0x7f5b44106470 2024-03-24T12:04:12.359 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.472+0000 7f5b4946a700 1 --2- 172.21.15.6:0/1431833931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b44106470 0x7f5b44105cf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00b7b0 tx=0x7f5b3c003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:12.359 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.472+0000 7f5b427fc700 1 -- 172.21.15.6:0/1431833931 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b3c00e040 con 0x7f5b44106470 2024-03-24T12:04:12.359 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.472+0000 7f5b427fc700 1 -- 172.21.15.6:0/1431833931 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f5b3c0094d0 con 0x7f5b44106470 2024-03-24T12:04:12.359 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.472+0000 7f5b4b6ce700 1 -- 172.21.15.6:0/1431833931 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b44104420 con 0x7f5b44106470 2024-03-24T12:04:12.359 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.472+0000 7f5b427fc700 1 -- 172.21.15.6:0/1431833931 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b3c0127a0 con 0x7f5b44106470 2024-03-24T12:04:12.359 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.472+0000 7f5b4b6ce700 1 -- 172.21.15.6:0/1431833931 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b44104920 con 0x7f5b44106470 2024-03-24T12:04:12.359 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.473+0000 7f5b427fc700 1 -- 172.21.15.6:0/1431833931 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) v1 ==== 45758+0+0 (secure 0 0 0) 0x7f5b3c01c650 con 0x7f5b44106470 2024-03-24T12:04:12.359 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.473+0000 7f5b4b6ce700 1 -- 172.21.15.6:0/1431833931 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b44109a70 con 0x7f5b44106470 2024-03-24T12:04:12.359 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.473+0000 7f5b427fc700 1 --2- 172.21.15.6:0/1431833931 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f5b2c038c60 0x7f5b2c03b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:12.359 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.473+0000 7f5b427fc700 1 -- 172.21.15.6:0/1431833931 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5b3c04cb40 con 0x7f5b44106470 2024-03-24T12:04:12.360 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.473+0000 7f5b48c69700 1 --2- 172.21.15.6:0/1431833931 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f5b2c038c60 0x7f5b2c03b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:12.360 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.474+0000 7f5b48c69700 1 --2- 172.21.15.6:0/1431833931 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f5b2c038c60 0x7f5b2c03b120 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5b38006fd0 tx=0x7f5b38006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:12.360 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.478+0000 7f5b427fc700 1 -- 172.21.15.6:0/1431833931 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5b3c003e60 con 0x7f5b44106470 2024-03-24T12:04:12.360 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.738+0000 7f5b4b6ce700 1 -- 172.21.15.6:0/1431833931 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f5b44109c80 con 0x7f5b44106470 2024-03-24T12:04:12.360 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.789+0000 7f5b427fc700 1 -- 172.21.15.6:0/1431833931 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f5b3c020020 con 0x7f5b44106470 2024-03-24T12:04:12.360 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.791+0000 7f5b427fc700 1 -- 172.21.15.6:0/1431833931 <== mon.0 v2:172.21.15.6:3300/0 8 ==== mgrmap(e 5) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f5b3c0129c0 con 0x7f5b44106470 2024-03-24T12:04:12.360 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.792+0000 7f5b4b6ce700 1 -- 172.21.15.6:0/1431833931 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f5b2c038c60 msgr2=0x7f5b2c03b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:12.360 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.792+0000 7f5b4b6ce700 1 --2- 172.21.15.6:0/1431833931 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f5b2c038c60 0x7f5b2c03b120 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5b38006fd0 tx=0x7f5b38006e40 comp rx=0 tx=0).stop 2024-03-24T12:04:12.360 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.792+0000 7f5b4b6ce700 1 -- 172.21.15.6:0/1431833931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b44106470 msgr2=0x7f5b44105cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:12.360 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.792+0000 7f5b4b6ce700 1 --2- 172.21.15.6:0/1431833931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b44106470 0x7f5b44105cf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00b7b0 tx=0x7f5b3c003d00 comp rx=0 tx=0).stop 2024-03-24T12:04:12.360 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.792+0000 7f5b4b6ce700 1 -- 172.21.15.6:0/1431833931 shutdown_connections 2024-03-24T12:04:12.360 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.792+0000 7f5b4b6ce700 1 --2- 172.21.15.6:0/1431833931 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f5b2c038c60 0x7f5b2c03b120 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:12.360 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.792+0000 7f5b4b6ce700 1 --2- 172.21.15.6:0/1431833931 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5b44106470 0x7f5b44105cf0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:12.360 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.792+0000 7f5b4b6ce700 1 -- 172.21.15.6:0/1431833931 >> 172.21.15.6:0/1431833931 conn(0x7f5b440ffec0 msgr2=0x7f5b441009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:12.361 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.792+0000 7f5b4b6ce700 1 -- 172.21.15.6:0/1431833931 shutdown_connections 2024-03-24T12:04:12.361 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:11.792+0000 7f5b4b6ce700 1 -- 172.21.15.6:0/1431833931 wait complete. 2024-03-24T12:04:12.602 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:12.359+0000 7feeeed1b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T12:04:12.602 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:12.458+0000 7feeeed1b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T12:04:12.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:12 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1431833931' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-24T12:04:12.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:12 smithi006 ceph-mon[30108]: mgrmap e5: y(active, since 6s) 2024-03-24T12:04:13.378 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:13 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:13.139+0000 7feeeed1b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T12:04:13.378 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:13 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:13.234+0000 7feeeed1b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T12:04:13.378 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:13 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:13.378+0000 7feeeed1b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T12:04:13.672 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:13 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:13.483+0000 7feeeed1b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T12:04:13.673 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:13 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:13.568+0000 7feeeed1b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T12:04:13.682 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-03-24T12:04:13.682 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-24T12:04:13.682 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-24T12:04:13.682 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-03-24T12:04:13.682 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-24T12:04:13.682 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-03-24T12:04:13.682 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.814+0000 7f0eedd52700 1 Processor -- start 2024-03-24T12:04:13.682 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.814+0000 7f0eedd52700 1 -- start start 2024-03-24T12:04:13.682 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.814+0000 7f0eedd52700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ee8106500 0x7f0ee81088f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:13.682 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.814+0000 7f0eedd52700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee80796c0 con 0x7f0ee8106500 2024-03-24T12:04:13.683 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.814+0000 7f0ee77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ee8106500 0x7f0ee81088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:13.683 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.814+0000 7f0ee77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ee8106500 0x7f0ee81088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35962/0 (socket says 172.21.15.6:35962) 2024-03-24T12:04:13.683 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.814+0000 7f0ee77fe700 1 -- 172.21.15.6:0/4000488281 learned_addr learned my addr 172.21.15.6:0/4000488281 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:13.683 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.814+0000 7f0ee77fe700 1 -- 172.21.15.6:0/4000488281 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee8108e30 con 0x7f0ee8106500 2024-03-24T12:04:13.683 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.815+0000 7f0ee77fe700 1 --2- 172.21.15.6:0/4000488281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ee8106500 0x7f0ee81088f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80098a0 tx=0x7f0ed8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f419fb23db0f339 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:13.683 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.815+0000 7f0ee67fc700 1 -- 172.21.15.6:0/4000488281 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ed8004160 con 0x7f0ee8106500 2024-03-24T12:04:13.683 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.815+0000 7f0ee67fc700 1 -- 172.21.15.6:0/4000488281 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f0ed80042c0 con 0x7f0ee8106500 2024-03-24T12:04:13.683 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.815+0000 7f0ee67fc700 1 -- 172.21.15.6:0/4000488281 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ed80118a0 con 0x7f0ee8106500 2024-03-24T12:04:13.683 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.815+0000 7f0eedd52700 1 -- 172.21.15.6:0/4000488281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ee8106500 msgr2=0x7f0ee81088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:13.683 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.815+0000 7f0eedd52700 1 --2- 172.21.15.6:0/4000488281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ee8106500 0x7f0ee81088f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80098a0 tx=0x7f0ed8009bb0 comp rx=0 tx=0).stop 2024-03-24T12:04:13.683 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.815+0000 7f0eedd52700 1 -- 172.21.15.6:0/4000488281 shutdown_connections 2024-03-24T12:04:13.683 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.815+0000 7f0eedd52700 1 --2- 172.21.15.6:0/4000488281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ee8106500 0x7f0ee81088f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:13.683 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.815+0000 7f0eedd52700 1 -- 172.21.15.6:0/4000488281 >> 172.21.15.6:0/4000488281 conn(0x7f0ee80ffef0 msgr2=0x7f0ee8102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:13.683 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.815+0000 7f0eedd52700 1 -- 172.21.15.6:0/4000488281 shutdown_connections 2024-03-24T12:04:13.684 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.816+0000 7f0eedd52700 1 -- 172.21.15.6:0/4000488281 wait complete. 2024-03-24T12:04:13.684 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.816+0000 7f0eedd52700 1 Processor -- start 2024-03-24T12:04:13.684 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.816+0000 7f0eedd52700 1 -- start start 2024-03-24T12:04:13.684 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.816+0000 7f0eedd52700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ee8106500 0x7f0ee8199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:13.684 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.816+0000 7f0eedd52700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee8199b80 con 0x7f0ee8106500 2024-03-24T12:04:13.684 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.817+0000 7f0ee77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ee8106500 0x7f0ee8199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:13.684 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.817+0000 7f0ee77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ee8106500 0x7f0ee8199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35968/0 (socket says 172.21.15.6:35968) 2024-03-24T12:04:13.684 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.817+0000 7f0ee77fe700 1 -- 172.21.15.6:0/2557225852 learned_addr learned my addr 172.21.15.6:0/2557225852 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:13.684 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.817+0000 7f0ee77fe700 1 -- 172.21.15.6:0/2557225852 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed8009550 con 0x7f0ee8106500 2024-03-24T12:04:13.684 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.817+0000 7f0ee77fe700 1 --2- 172.21.15.6:0/2557225852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ee8106500 0x7f0ee8199640 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8003da0 tx=0x7f0ed8011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:13.684 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.817+0000 7f0ee4ff9700 1 -- 172.21.15.6:0/2557225852 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ed8011930 con 0x7f0ee8106500 2024-03-24T12:04:13.684 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.817+0000 7f0eedd52700 1 -- 172.21.15.6:0/2557225852 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee8199d80 con 0x7f0ee8106500 2024-03-24T12:04:13.684 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.817+0000 7f0ee4ff9700 1 -- 172.21.15.6:0/2557225852 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f0ed8003b00 con 0x7f0ee8106500 2024-03-24T12:04:13.684 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.817+0000 7f0ee4ff9700 1 -- 172.21.15.6:0/2557225852 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ed801ad50 con 0x7f0ee8106500 2024-03-24T12:04:13.685 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.817+0000 7f0eedd52700 1 -- 172.21.15.6:0/2557225852 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee819a220 con 0x7f0ee8106500 2024-03-24T12:04:13.685 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.818+0000 7f0ee4ff9700 1 -- 172.21.15.6:0/2557225852 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f0ed802d440 con 0x7f0ee8106500 2024-03-24T12:04:13.685 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.818+0000 7f0ee4ff9700 1 --2- 172.21.15.6:0/2557225852 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f0ed0038c60 0x7f0ed003b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:13.685 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.818+0000 7f0ee4ff9700 1 -- 172.21.15.6:0/2557225852 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0ed804cbd0 con 0x7f0ee8106500 2024-03-24T12:04:13.685 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.818+0000 7f0eedd52700 1 -- 172.21.15.6:0/2557225852 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee8109aa0 con 0x7f0ee8106500 2024-03-24T12:04:13.685 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.818+0000 7f0ee6ffd700 1 -- 172.21.15.6:0/2557225852 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f0ed0038c60 msgr2=0x7f0ed003b120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/2054386689 2024-03-24T12:04:13.685 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.818+0000 7f0ee6ffd700 1 --2- 172.21.15.6:0/2557225852 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f0ed0038c60 0x7f0ed003b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T12:04:13.685 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:12.824+0000 7f0ee4ff9700 1 -- 172.21.15.6:0/2557225852 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0ed804ce60 con 0x7f0ee8106500 2024-03-24T12:04:13.685 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:13.019+0000 7f0ee6ffd700 1 -- 172.21.15.6:0/2557225852 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f0ed0038c60 msgr2=0x7f0ed003b120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/2054386689 2024-03-24T12:04:13.685 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:13.019+0000 7f0ee6ffd700 1 --2- 172.21.15.6:0/2557225852 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f0ed0038c60 0x7f0ed003b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-24T12:04:13.685 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:13.092+0000 7f0eedd52700 1 -- 172.21.15.6:0/2557225852 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f0ee819cc50 con 0x7f0ee8106500 2024-03-24T12:04:13.685 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:13.092+0000 7f0ee4ff9700 1 -- 172.21.15.6:0/2557225852 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f0ed802d740 con 0x7f0ee8106500 2024-03-24T12:04:13.685 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:13.094+0000 7f0eedd52700 1 -- 172.21.15.6:0/2557225852 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f0ed0038c60 msgr2=0x7f0ed003b120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T12:04:13.686 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:13.094+0000 7f0eedd52700 1 --2- 172.21.15.6:0/2557225852 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f0ed0038c60 0x7f0ed003b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:13.686 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:13.094+0000 7f0eedd52700 1 -- 172.21.15.6:0/2557225852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ee8106500 msgr2=0x7f0ee8199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:13.686 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:13.094+0000 7f0eedd52700 1 --2- 172.21.15.6:0/2557225852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ee8106500 0x7f0ee8199640 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8003da0 tx=0x7f0ed8011e90 comp rx=0 tx=0).stop 2024-03-24T12:04:13.686 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:13.094+0000 7f0eedd52700 1 -- 172.21.15.6:0/2557225852 shutdown_connections 2024-03-24T12:04:13.686 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:13.094+0000 7f0eedd52700 1 --2- 172.21.15.6:0/2557225852 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7f0ed0038c60 0x7f0ed003b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:13.686 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:13.094+0000 7f0eedd52700 1 --2- 172.21.15.6:0/2557225852 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ee8106500 0x7f0ee8199640 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:13.686 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:13.094+0000 7f0eedd52700 1 -- 172.21.15.6:0/2557225852 >> 172.21.15.6:0/2557225852 conn(0x7f0ee80ffef0 msgr2=0x7f0ee81009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:13.686 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:13.095+0000 7f0eedd52700 1 -- 172.21.15.6:0/2557225852 shutdown_connections 2024-03-24T12:04:13.686 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:13.095+0000 7f0eedd52700 1 -- 172.21.15.6:0/2557225852 wait complete. 2024-03-24T12:04:13.686 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for the mgr to restart... 2024-03-24T12:04:13.686 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr epoch 5... 2024-03-24T12:04:13.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:13 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2557225852' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-24T12:04:13.961 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:13 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:13.846+0000 7feeeed1b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-24T12:04:14.333 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:13 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:13.965+0000 7feeeed1b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T12:04:14.333 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:14 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:14.055+0000 7feeeed1b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T12:04:14.333 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:14 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:14.155+0000 7feeeed1b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T12:04:14.333 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:14 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:14.333+0000 7feeeed1b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T12:04:15.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:15.317+0000 7feeeed1b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T12:04:15.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:15.450+0000 7feeeed1b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T12:04:16.078 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:15.787+0000 7feeeed1b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T12:04:16.078 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:15.879+0000 7feeeed1b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T12:04:16.078 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:16 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:16.078+0000 7feeeed1b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T12:04:16.461 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:16 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:16.267+0000 7feeeed1b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T12:04:16.817 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:16 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:16.557+0000 7feeeed1b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T12:04:17.138 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:16 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:16.818+0000 7feeeed1b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T12:04:17.138 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:17 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:17.059+0000 7feeeed1b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T12:04:18.211 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:17 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:17.877+0000 7feeeed1b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T12:04:18.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:18 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:18.459+0000 7feeeed1b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T12:04:18.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:18 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:18.559+0000 7feeeed1b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T12:04:19.211 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:18 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:18.778+0000 7feeeed1b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T12:04:19.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:18 smithi006 ceph-mon[30108]: Active manager daemon y restarted 2024-03-24T12:04:19.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:18 smithi006 ceph-mon[30108]: Activating manager daemon y 2024-03-24T12:04:19.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:18 smithi006 ceph-mon[30108]: osdmap e2: 0 total, 0 up, 0 in 2024-03-24T12:04:19.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:18 smithi006 ceph-mon[30108]: mgrmap e6: y(active, starting, since 0.00335521s) 2024-03-24T12:04:19.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:18 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T12:04:19.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:18 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T12:04:19.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:18 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T12:04:19.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:18 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T12:04:19.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:18 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T12:04:19.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:18 smithi006 ceph-mon[30108]: Manager daemon y is now available 2024-03-24T12:04:19.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:18 smithi006 ceph-mon[30108]: Found migration_current of "None". Setting to last migration. 2024-03-24T12:04:19.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:18 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' 2024-03-24T12:04:19.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:18 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' 2024-03-24T12:04:19.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:18 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:04:19.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:18 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:04:20.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:19 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:04:20.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:19 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T12:04:20.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:19 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T12:04:20.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:19 smithi006 ceph-mon[30108]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-24T12:04:20.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:19 smithi006 ceph-mon[30108]: mgrmap e7: y(active, since 1.00545s) 2024-03-24T12:04:20.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:19 smithi006 ceph-mon[30108]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-24T12:04:20.381 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-03-24T12:04:20.381 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-24T12:04:20.381 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-24T12:04:20.381 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-03-24T12:04:20.381 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.130+0000 7fcaf8d71700 1 Processor -- start 2024-03-24T12:04:20.381 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.130+0000 7fcaf8d71700 1 -- start start 2024-03-24T12:04:20.381 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.130+0000 7fcaf8d71700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcaf4106560 0x7fcaf4108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:20.381 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.130+0000 7fcaf8d71700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf4079660 con 0x7fcaf4106560 2024-03-24T12:04:20.381 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.130+0000 7fcaf259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcaf4106560 0x7fcaf4108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:20.381 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.130+0000 7fcaf259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcaf4106560 0x7fcaf4108950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35974/0 (socket says 172.21.15.6:35974) 2024-03-24T12:04:20.381 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.130+0000 7fcaf259c700 1 -- 172.21.15.6:0/1368483856 learned_addr learned my addr 172.21.15.6:0/1368483856 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:20.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.130+0000 7fcaf259c700 1 -- 172.21.15.6:0/1368483856 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf40797a0 con 0x7fcaf4106560 2024-03-24T12:04:20.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.131+0000 7fcaf259c700 1 --2- 172.21.15.6:0/1368483856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcaf4106560 0x7fcaf4108950 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcae40098a0 tx=0x7fcae4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa80b4f125f0b9b server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:20.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.131+0000 7fcaf159a700 1 -- 172.21.15.6:0/1368483856 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcae4004160 con 0x7fcaf4106560 2024-03-24T12:04:20.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.131+0000 7fcaf159a700 1 -- 172.21.15.6:0/1368483856 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fcae40042c0 con 0x7fcaf4106560 2024-03-24T12:04:20.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.131+0000 7fcaf159a700 1 -- 172.21.15.6:0/1368483856 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcae40118a0 con 0x7fcaf4106560 2024-03-24T12:04:20.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.131+0000 7fcaf8d71700 1 -- 172.21.15.6:0/1368483856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcaf4106560 msgr2=0x7fcaf4108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:20.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.131+0000 7fcaf8d71700 1 --2- 172.21.15.6:0/1368483856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcaf4106560 0x7fcaf4108950 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcae40098a0 tx=0x7fcae4009bb0 comp rx=0 tx=0).stop 2024-03-24T12:04:20.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.132+0000 7fcaf8d71700 1 -- 172.21.15.6:0/1368483856 shutdown_connections 2024-03-24T12:04:20.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.132+0000 7fcaf8d71700 1 --2- 172.21.15.6:0/1368483856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcaf4106560 0x7fcaf4108950 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:20.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.132+0000 7fcaf8d71700 1 -- 172.21.15.6:0/1368483856 >> 172.21.15.6:0/1368483856 conn(0x7fcaf40ffef0 msgr2=0x7fcaf4102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:20.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.132+0000 7fcaf8d71700 1 -- 172.21.15.6:0/1368483856 shutdown_connections 2024-03-24T12:04:20.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.132+0000 7fcaf8d71700 1 -- 172.21.15.6:0/1368483856 wait complete. 2024-03-24T12:04:20.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.132+0000 7fcaf8d71700 1 Processor -- start 2024-03-24T12:04:20.382 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.132+0000 7fcaf8d71700 1 -- start start 2024-03-24T12:04:20.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.133+0000 7fcaf8d71700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcaf4106560 0x7fcaf4105de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:20.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.133+0000 7fcaf8d71700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf4106320 con 0x7fcaf4106560 2024-03-24T12:04:20.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.133+0000 7fcaf259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcaf4106560 0x7fcaf4105de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:20.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.133+0000 7fcaf259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcaf4106560 0x7fcaf4105de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35976/0 (socket says 172.21.15.6:35976) 2024-03-24T12:04:20.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.133+0000 7fcaf259c700 1 -- 172.21.15.6:0/281607166 learned_addr learned my addr 172.21.15.6:0/281607166 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:20.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.133+0000 7fcaf259c700 1 -- 172.21.15.6:0/281607166 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae4009550 con 0x7fcaf4106560 2024-03-24T12:04:20.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.133+0000 7fcaf259c700 1 --2- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcaf4106560 0x7fcaf4105de0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcae4011890 tx=0x7fcae4011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:20.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.133+0000 7fcaeb7fe700 1 -- 172.21.15.6:0/281607166 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcae40036a0 con 0x7fcaf4106560 2024-03-24T12:04:20.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.133+0000 7fcaeb7fe700 1 -- 172.21.15.6:0/281607166 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fcae401a430 con 0x7fcaf4106560 2024-03-24T12:04:20.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.133+0000 7fcaf8d71700 1 -- 172.21.15.6:0/281607166 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf4104510 con 0x7fcaf4106560 2024-03-24T12:04:20.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.133+0000 7fcaeb7fe700 1 -- 172.21.15.6:0/281607166 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcae40193c0 con 0x7fcaf4106560 2024-03-24T12:04:20.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.133+0000 7fcaf8d71700 1 -- 172.21.15.6:0/281607166 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf41049b0 con 0x7fcaf4106560 2024-03-24T12:04:20.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.134+0000 7fcaeb7fe700 1 -- 172.21.15.6:0/281607166 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fcae40195f0 con 0x7fcaf4106560 2024-03-24T12:04:20.383 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.134+0000 7fcaeb7fe700 1 --2- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7fcae0038d00 0x7fcae003b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:20.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.134+0000 7fcaeb7fe700 1 -- 172.21.15.6:0/281607166 --> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcae003b8d0 con 0x7fcae0038d00 2024-03-24T12:04:20.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.134+0000 7fcaeb7fe700 1 -- 172.21.15.6:0/281607166 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcae404cc00 con 0x7fcaf4106560 2024-03-24T12:04:20.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.134+0000 7fcaf1d9b700 1 -- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7fcae0038d00 msgr2=0x7fcae003b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/2054386689 2024-03-24T12:04:20.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.134+0000 7fcaf1d9b700 1 --2- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7fcae0038d00 0x7fcae003b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T12:04:20.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.335+0000 7fcaf1d9b700 1 -- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7fcae0038d00 msgr2=0x7fcae003b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/2054386689 2024-03-24T12:04:20.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.335+0000 7fcaf1d9b700 1 --2- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7fcae0038d00 0x7fcae003b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-24T12:04:20.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.736+0000 7fcaf1d9b700 1 -- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7fcae0038d00 msgr2=0x7fcae003b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/2054386689 2024-03-24T12:04:20.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:14.736+0000 7fcaf1d9b700 1 --2- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7fcae0038d00 0x7fcae003b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-24T12:04:20.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:15.537+0000 7fcaf1d9b700 1 -- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7fcae0038d00 msgr2=0x7fcae003b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/2054386689 2024-03-24T12:04:20.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:15.537+0000 7fcaf1d9b700 1 --2- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7fcae0038d00 0x7fcae003b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-24T12:04:20.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:17.138+0000 7fcaf1d9b700 1 -- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7fcae0038d00 msgr2=0x7fcae003b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/2054386689 2024-03-24T12:04:20.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:17.138+0000 7fcaf1d9b700 1 --2- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7fcae0038d00 0x7fcae003b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-24T12:04:20.384 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:18.794+0000 7fcaeb7fe700 1 -- 172.21.15.6:0/281607166 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mgrmap(e 6) v1 ==== 45536+0+0 (secure 0 0 0) 0x7fcae401a8d0 con 0x7fcaf4106560 2024-03-24T12:04:20.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:18.794+0000 7fcaeb7fe700 1 -- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7fcae0038d00 msgr2=0x7fcae003b1c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T12:04:20.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:18.794+0000 7fcaeb7fe700 1 --2- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7fcae0038d00 0x7fcae003b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:20.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:19.795+0000 7fcaeb7fe700 1 -- 172.21.15.6:0/281607166 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 7) v1 ==== 45663+0+0 (secure 0 0 0) 0x7fcae404e5d0 con 0x7fcaf4106560 2024-03-24T12:04:20.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:19.795+0000 7fcaeb7fe700 1 --2- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7fcae003c5f0 0x7fcae003e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:20.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:19.795+0000 7fcaeb7fe700 1 -- 172.21.15.6:0/281607166 --> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcae003b8d0 con 0x7fcae003c5f0 2024-03-24T12:04:20.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:19.796+0000 7fcaf1d9b700 1 --2- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7fcae003c5f0 0x7fcae003e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:20.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:19.796+0000 7fcaf1d9b700 1 --2- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7fcae003c5f0 0x7fcae003e9e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fcadc003a10 tx=0x7fcadc0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:20.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:19.797+0000 7fcaeb7fe700 1 -- 172.21.15.6:0/281607166 <== mgr.14120 v2:172.21.15.6:6800/3010671792 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fcae003b8d0 con 0x7fcae003c5f0 2024-03-24T12:04:20.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:19.804+0000 7fcaf8d71700 1 -- 172.21.15.6:0/281607166 --> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fcaf40632a0 con 0x7fcae003c5f0 2024-03-24T12:04:20.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:19.804+0000 7fcaeb7fe700 1 -- 172.21.15.6:0/281607166 <== mgr.14120 v2:172.21.15.6:6800/3010671792 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fcaf40632a0 con 0x7fcae003c5f0 2024-03-24T12:04:20.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:19.804+0000 7fcaf8d71700 1 -- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7fcae003c5f0 msgr2=0x7fcae003e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:20.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:19.804+0000 7fcaf8d71700 1 --2- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7fcae003c5f0 0x7fcae003e9e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fcadc003a10 tx=0x7fcadc0092b0 comp rx=0 tx=0).stop 2024-03-24T12:04:20.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:19.804+0000 7fcaf8d71700 1 -- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcaf4106560 msgr2=0x7fcaf4105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:20.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:19.804+0000 7fcaf8d71700 1 --2- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcaf4106560 0x7fcaf4105de0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcae4011890 tx=0x7fcae4011970 comp rx=0 tx=0).stop 2024-03-24T12:04:20.385 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:19.804+0000 7fcaf8d71700 1 -- 172.21.15.6:0/281607166 shutdown_connections 2024-03-24T12:04:20.386 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:19.804+0000 7fcaf8d71700 1 --2- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7fcae003c5f0 0x7fcae003e9e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:20.386 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:19.804+0000 7fcaf8d71700 1 --2- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcaf4106560 0x7fcaf4105de0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:20.386 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:19.804+0000 7fcaf8d71700 1 --2- 172.21.15.6:0/281607166 >> [v2:172.21.15.6:6800/2054386689,v1:172.21.15.6:6801/2054386689] conn(0x7fcae0038d00 0x7fcae003b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:20.386 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:19.804+0000 7fcaf8d71700 1 -- 172.21.15.6:0/281607166 >> 172.21.15.6:0/281607166 conn(0x7fcaf40ffef0 msgr2=0x7fcaf41009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:20.386 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:19.804+0000 7fcaf8d71700 1 -- 172.21.15.6:0/281607166 shutdown_connections 2024-03-24T12:04:20.386 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:19.804+0000 7fcaf8d71700 1 -- 172.21.15.6:0/281607166 wait complete. 2024-03-24T12:04:20.386 INFO:teuthology.orchestra.run.smithi006.stdout:mgr epoch 5 is available 2024-03-24T12:04:20.386 INFO:teuthology.orchestra.run.smithi006.stdout:Setting orchestrator backend to cephadm... 2024-03-24T12:04:21.385 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:20 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' 2024-03-24T12:04:21.385 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:20 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' 2024-03-24T12:04:21.385 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:20 smithi006 ceph-mon[30108]: [24/Mar/2024:12:04:20] ENGINE Bus STARTING 2024-03-24T12:04:21.386 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:20 smithi006 ceph-mon[30108]: [24/Mar/2024:12:04:20] ENGINE Serving on https://172.21.15.6:7150 2024-03-24T12:04:21.386 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:20 smithi006 ceph-mon[30108]: [24/Mar/2024:12:04:20] ENGINE Serving on http://172.21.15.6:8765 2024-03-24T12:04:21.386 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:20 smithi006 ceph-mon[30108]: [24/Mar/2024:12:04:20] ENGINE Bus STARTED 2024-03-24T12:04:21.386 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:20 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:04:21.652 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.845+0000 7f6364444700 1 Processor -- start 2024-03-24T12:04:21.652 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.845+0000 7f6364444700 1 -- start start 2024-03-24T12:04:21.652 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.845+0000 7f6364444700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635c1064a0 0x7f635c108890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:21.652 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.845+0000 7f6364444700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635c079720 con 0x7f635c1064a0 2024-03-24T12:04:21.652 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.845+0000 7f63621e0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635c1064a0 0x7f635c108890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:21.652 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.845+0000 7f63621e0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635c1064a0 0x7f635c108890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42294/0 (socket says 172.21.15.6:42294) 2024-03-24T12:04:21.652 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.845+0000 7f63621e0700 1 -- 172.21.15.6:0/4192937718 learned_addr learned my addr 172.21.15.6:0/4192937718 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:21.652 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.845+0000 7f63621e0700 1 -- 172.21.15.6:0/4192937718 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635c108dd0 con 0x7f635c1064a0 2024-03-24T12:04:21.652 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.846+0000 7f63621e0700 1 --2- 172.21.15.6:0/4192937718 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635c1064a0 0x7f635c108890 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f63580098a0 tx=0x7f6358009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf00b7ec3456530a server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:21.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.846+0000 7f63611de700 1 -- 172.21.15.6:0/4192937718 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6358004160 con 0x7f635c1064a0 2024-03-24T12:04:21.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.846+0000 7f63611de700 1 -- 172.21.15.6:0/4192937718 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f63580042c0 con 0x7f635c1064a0 2024-03-24T12:04:21.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.846+0000 7f63611de700 1 -- 172.21.15.6:0/4192937718 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63580118a0 con 0x7f635c1064a0 2024-03-24T12:04:21.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.846+0000 7f6364444700 1 -- 172.21.15.6:0/4192937718 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635c1064a0 msgr2=0x7f635c108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:21.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.846+0000 7f6364444700 1 --2- 172.21.15.6:0/4192937718 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635c1064a0 0x7f635c108890 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f63580098a0 tx=0x7f6358009bb0 comp rx=0 tx=0).stop 2024-03-24T12:04:21.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.847+0000 7f6364444700 1 -- 172.21.15.6:0/4192937718 shutdown_connections 2024-03-24T12:04:21.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.847+0000 7f6364444700 1 --2- 172.21.15.6:0/4192937718 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635c1064a0 0x7f635c108890 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:21.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.847+0000 7f6364444700 1 -- 172.21.15.6:0/4192937718 >> 172.21.15.6:0/4192937718 conn(0x7f635c0ffef0 msgr2=0x7f635c102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:21.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.847+0000 7f6364444700 1 -- 172.21.15.6:0/4192937718 shutdown_connections 2024-03-24T12:04:21.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.847+0000 7f6364444700 1 -- 172.21.15.6:0/4192937718 wait complete. 2024-03-24T12:04:21.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.847+0000 7f6364444700 1 Processor -- start 2024-03-24T12:04:21.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.847+0000 7f6364444700 1 -- start start 2024-03-24T12:04:21.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.847+0000 7f6364444700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635c1064a0 0x7f635c199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:21.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.847+0000 7f6364444700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635c199ba0 con 0x7f635c1064a0 2024-03-24T12:04:21.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.848+0000 7f63621e0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635c1064a0 0x7f635c199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:21.653 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.848+0000 7f63621e0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635c1064a0 0x7f635c199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42302/0 (socket says 172.21.15.6:42302) 2024-03-24T12:04:21.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.848+0000 7f63621e0700 1 -- 172.21.15.6:0/710121493 learned_addr learned my addr 172.21.15.6:0/710121493 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:21.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.848+0000 7f63621e0700 1 -- 172.21.15.6:0/710121493 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6358009550 con 0x7f635c1064a0 2024-03-24T12:04:21.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.848+0000 7f63621e0700 1 --2- 172.21.15.6:0/710121493 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635c1064a0 0x7f635c199660 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6358003da0 tx=0x7f6358011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:21.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.848+0000 7f63537fe700 1 -- 172.21.15.6:0/710121493 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6358011930 con 0x7f635c1064a0 2024-03-24T12:04:21.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.848+0000 7f63537fe700 1 -- 172.21.15.6:0/710121493 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f6358003b00 con 0x7f635c1064a0 2024-03-24T12:04:21.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.848+0000 7f63537fe700 1 -- 172.21.15.6:0/710121493 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f635801ad50 con 0x7f635c1064a0 2024-03-24T12:04:21.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.848+0000 7f6364444700 1 -- 172.21.15.6:0/710121493 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f635c199da0 con 0x7f635c1064a0 2024-03-24T12:04:21.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.848+0000 7f6364444700 1 -- 172.21.15.6:0/710121493 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f635c19a240 con 0x7f635c1064a0 2024-03-24T12:04:21.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.849+0000 7f63537fe700 1 -- 172.21.15.6:0/710121493 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 7) v1 ==== 45663+0+0 (secure 0 0 0) 0x7f635802d440 con 0x7f635c1064a0 2024-03-24T12:04:21.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.849+0000 7f63537fe700 1 --2- 172.21.15.6:0/710121493 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f6348038ba0 0x7f634803b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:21.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.849+0000 7f6364444700 1 -- 172.21.15.6:0/710121493 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f635c109aa0 con 0x7f635c1064a0 2024-03-24T12:04:21.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.849+0000 7f63537fe700 1 -- 172.21.15.6:0/710121493 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f635804cb40 con 0x7f635c1064a0 2024-03-24T12:04:21.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.849+0000 7f63619df700 1 --2- 172.21.15.6:0/710121493 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f6348038ba0 0x7f634803b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:21.654 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.850+0000 7f63619df700 1 --2- 172.21.15.6:0/710121493 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f6348038ba0 0x7f634803b060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f634c006fd0 tx=0x7f634c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:21.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.854+0000 7f63537fe700 1 -- 172.21.15.6:0/710121493 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f635807cd20 con 0x7f635c1064a0 2024-03-24T12:04:21.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:20.983+0000 7f63537fe700 1 -- 172.21.15.6:0/710121493 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f635801a630 con 0x7f635c1064a0 2024-03-24T12:04:21.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:21.075+0000 7f6364444700 1 -- 172.21.15.6:0/710121493 --> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f635c101140 con 0x7f6348038ba0 2024-03-24T12:04:21.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:21.080+0000 7f63537fe700 1 -- 172.21.15.6:0/710121493 <== mgr.14120 v2:172.21.15.6:6800/3010671792 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f635c101140 con 0x7f6348038ba0 2024-03-24T12:04:21.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:21.082+0000 7f6364444700 1 -- 172.21.15.6:0/710121493 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f6348038ba0 msgr2=0x7f634803b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:21.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:21.082+0000 7f6364444700 1 --2- 172.21.15.6:0/710121493 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f6348038ba0 0x7f634803b060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f634c006fd0 tx=0x7f634c006e40 comp rx=0 tx=0).stop 2024-03-24T12:04:21.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:21.082+0000 7f6364444700 1 -- 172.21.15.6:0/710121493 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635c1064a0 msgr2=0x7f635c199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:21.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:21.082+0000 7f6364444700 1 --2- 172.21.15.6:0/710121493 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635c1064a0 0x7f635c199660 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6358003da0 tx=0x7f6358011e90 comp rx=0 tx=0).stop 2024-03-24T12:04:21.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:21.082+0000 7f6364444700 1 -- 172.21.15.6:0/710121493 shutdown_connections 2024-03-24T12:04:21.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:21.082+0000 7f6364444700 1 --2- 172.21.15.6:0/710121493 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f6348038ba0 0x7f634803b060 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:21.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:21.082+0000 7f6364444700 1 --2- 172.21.15.6:0/710121493 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f635c1064a0 0x7f635c199660 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:21.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:21.082+0000 7f6364444700 1 -- 172.21.15.6:0/710121493 >> 172.21.15.6:0/710121493 conn(0x7f635c0ffef0 msgr2=0x7f635c100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:21.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:21.082+0000 7f6364444700 1 -- 172.21.15.6:0/710121493 shutdown_connections 2024-03-24T12:04:21.655 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:21.082+0000 7f6364444700 1 -- 172.21.15.6:0/710121493 wait complete. 2024-03-24T12:04:22.315 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:21 smithi006 ceph-mon[30108]: mgrmap e8: y(active, since 2s) 2024-03-24T12:04:22.315 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:21 smithi006 ceph-mon[30108]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:04:22.315 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:21 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' 2024-03-24T12:04:22.315 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:21 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:04:22.917 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-24T12:04:22.918 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.103+0000 7f696827c700 1 Processor -- start 2024-03-24T12:04:22.918 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.104+0000 7f696827c700 1 -- start start 2024-03-24T12:04:22.918 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.104+0000 7f696827c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6960104930 0x7f6960104d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:22.918 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.104+0000 7f696827c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6960105320 con 0x7f6960104930 2024-03-24T12:04:22.918 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.104+0000 7f6966018700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6960104930 0x7f6960104d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:22.918 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.104+0000 7f6966018700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6960104930 0x7f6960104d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42304/0 (socket says 172.21.15.6:42304) 2024-03-24T12:04:22.918 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.104+0000 7f6966018700 1 -- 172.21.15.6:0/3520956830 learned_addr learned my addr 172.21.15.6:0/3520956830 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:22.918 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.104+0000 7f6966018700 1 -- 172.21.15.6:0/3520956830 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6960105b30 con 0x7f6960104930 2024-03-24T12:04:22.918 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.104+0000 7f6966018700 1 --2- 172.21.15.6:0/3520956830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6960104930 0x7f6960104d50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f69500098a0 tx=0x7f6950009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f896dc098e38485f server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:22.918 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.105+0000 7f6965016700 1 -- 172.21.15.6:0/3520956830 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6950004160 con 0x7f6960104930 2024-03-24T12:04:22.918 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.105+0000 7f6965016700 1 -- 172.21.15.6:0/3520956830 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f69500042c0 con 0x7f6960104930 2024-03-24T12:04:22.918 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.105+0000 7f6965016700 1 -- 172.21.15.6:0/3520956830 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69500118a0 con 0x7f6960104930 2024-03-24T12:04:22.919 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.105+0000 7f696827c700 1 -- 172.21.15.6:0/3520956830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6960104930 msgr2=0x7f6960104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:22.919 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.105+0000 7f696827c700 1 --2- 172.21.15.6:0/3520956830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6960104930 0x7f6960104d50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f69500098a0 tx=0x7f6950009bb0 comp rx=0 tx=0).stop 2024-03-24T12:04:22.919 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.105+0000 7f696827c700 1 -- 172.21.15.6:0/3520956830 shutdown_connections 2024-03-24T12:04:22.919 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.105+0000 7f696827c700 1 --2- 172.21.15.6:0/3520956830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6960104930 0x7f6960104d50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:22.919 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.105+0000 7f696827c700 1 -- 172.21.15.6:0/3520956830 >> 172.21.15.6:0/3520956830 conn(0x7f69600ffef0 msgr2=0x7f6960102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:22.919 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.105+0000 7f696827c700 1 -- 172.21.15.6:0/3520956830 shutdown_connections 2024-03-24T12:04:22.919 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.105+0000 7f696827c700 1 -- 172.21.15.6:0/3520956830 wait complete. 2024-03-24T12:04:22.919 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.106+0000 7f696827c700 1 Processor -- start 2024-03-24T12:04:22.919 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.106+0000 7f696827c700 1 -- start start 2024-03-24T12:04:22.919 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.106+0000 7f696827c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6960104930 0x7f69601996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:22.919 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.106+0000 7f696827c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6960199be0 con 0x7f6960104930 2024-03-24T12:04:22.919 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.106+0000 7f6966018700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6960104930 0x7f69601996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:22.919 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.106+0000 7f6966018700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6960104930 0x7f69601996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42318/0 (socket says 172.21.15.6:42318) 2024-03-24T12:04:22.919 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.106+0000 7f6966018700 1 -- 172.21.15.6:0/2028366056 learned_addr learned my addr 172.21.15.6:0/2028366056 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:22.920 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.106+0000 7f6966018700 1 -- 172.21.15.6:0/2028366056 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6950009550 con 0x7f6960104930 2024-03-24T12:04:22.920 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.107+0000 7f6966018700 1 --2- 172.21.15.6:0/2028366056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6960104930 0x7f69601996a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6950003da0 tx=0x7f6950011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:22.920 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.107+0000 7f69577fe700 1 -- 172.21.15.6:0/2028366056 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6950011930 con 0x7f6960104930 2024-03-24T12:04:22.920 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.107+0000 7f69577fe700 1 -- 172.21.15.6:0/2028366056 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f6950003b00 con 0x7f6960104930 2024-03-24T12:04:22.920 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.107+0000 7f696827c700 1 -- 172.21.15.6:0/2028366056 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6960199de0 con 0x7f6960104930 2024-03-24T12:04:22.920 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.107+0000 7f69577fe700 1 -- 172.21.15.6:0/2028366056 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f695001ad50 con 0x7f6960104930 2024-03-24T12:04:22.920 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.107+0000 7f696827c700 1 -- 172.21.15.6:0/2028366056 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f696019a280 con 0x7f6960104930 2024-03-24T12:04:22.920 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.108+0000 7f696827c700 1 -- 172.21.15.6:0/2028366056 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6960193800 con 0x7f6960104930 2024-03-24T12:04:22.920 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.108+0000 7f69577fe700 1 -- 172.21.15.6:0/2028366056 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f69500036a0 con 0x7f6960104930 2024-03-24T12:04:22.920 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.108+0000 7f69577fe700 1 --2- 172.21.15.6:0/2028366056 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f694c038cb0 0x7f694c03b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:22.920 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.108+0000 7f69577fe700 1 -- 172.21.15.6:0/2028366056 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f695004cb40 con 0x7f6960104930 2024-03-24T12:04:22.920 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.108+0000 7f6965817700 1 --2- 172.21.15.6:0/2028366056 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f694c038cb0 0x7f694c03b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:22.920 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.109+0000 7f6965817700 1 --2- 172.21.15.6:0/2028366056 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f694c038cb0 0x7f694c03b170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f695c006fd0 tx=0x7f695c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:22.920 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.113+0000 7f69577fe700 1 -- 172.21.15.6:0/2028366056 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f695001a630 con 0x7f6960104930 2024-03-24T12:04:22.921 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.315+0000 7f696827c700 1 -- 172.21.15.6:0/2028366056 --> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f696002d7d0 con 0x7f694c038cb0 2024-03-24T12:04:22.921 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.316+0000 7f69577fe700 1 -- 172.21.15.6:0/2028366056 <== mgr.14120 v2:172.21.15.6:6800/3010671792 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f696002d7d0 con 0x7f694c038cb0 2024-03-24T12:04:22.921 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.319+0000 7f696827c700 1 -- 172.21.15.6:0/2028366056 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f694c038cb0 msgr2=0x7f694c03b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:22.921 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.319+0000 7f696827c700 1 --2- 172.21.15.6:0/2028366056 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f694c038cb0 0x7f694c03b170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f695c006fd0 tx=0x7f695c006e40 comp rx=0 tx=0).stop 2024-03-24T12:04:22.921 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.319+0000 7f696827c700 1 -- 172.21.15.6:0/2028366056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6960104930 msgr2=0x7f69601996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:22.921 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.319+0000 7f696827c700 1 --2- 172.21.15.6:0/2028366056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6960104930 0x7f69601996a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6950003da0 tx=0x7f6950011e90 comp rx=0 tx=0).stop 2024-03-24T12:04:22.921 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.319+0000 7f696827c700 1 -- 172.21.15.6:0/2028366056 shutdown_connections 2024-03-24T12:04:22.921 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.319+0000 7f696827c700 1 --2- 172.21.15.6:0/2028366056 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f694c038cb0 0x7f694c03b170 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:22.921 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.319+0000 7f696827c700 1 --2- 172.21.15.6:0/2028366056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6960104930 0x7f69601996a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:22.921 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.319+0000 7f696827c700 1 -- 172.21.15.6:0/2028366056 >> 172.21.15.6:0/2028366056 conn(0x7f69600ffef0 msgr2=0x7f6960100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:22.922 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.319+0000 7f696827c700 1 -- 172.21.15.6:0/2028366056 shutdown_connections 2024-03-24T12:04:22.922 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:22.319+0000 7f696827c700 1 -- 172.21.15.6:0/2028366056 wait complete. 2024-03-24T12:04:22.922 INFO:teuthology.orchestra.run.smithi006.stdout:Generating ssh key... 2024-03-24T12:04:23.166 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:22 smithi006 ceph-mon[30108]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:04:23.818 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:23 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: Generating public/private rsa key pair. 2024-03-24T12:04:23.818 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:23 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: Your identification has been saved in /tmp/tmputob0zsq/key. 2024-03-24T12:04:23.818 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:23 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: Your public key has been saved in /tmp/tmputob0zsq/key.pub. 2024-03-24T12:04:23.818 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:23 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: The key fingerprint is: 2024-03-24T12:04:23.818 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:23 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: SHA256:oIrhHXmWkcQQKH7oaUOMxtRHGQyR22SZhiyKXlQwt8c ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715 2024-03-24T12:04:23.818 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:23 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: The key's randomart image is: 2024-03-24T12:04:23.818 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:23 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: +---[RSA 3072]----+ 2024-03-24T12:04:23.818 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:23 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: | +B&*= | 2024-03-24T12:04:23.818 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:23 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: |.+ *+@+ | 2024-03-24T12:04:23.818 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:23 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: |X + B+.E | 2024-03-24T12:04:23.819 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:23 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: |+B +.o+. | 2024-03-24T12:04:23.819 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:23 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: |* +o.+ S | 2024-03-24T12:04:23.819 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:23 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: |.O..+ | 2024-03-24T12:04:23.819 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:23 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: |o.o. | 2024-03-24T12:04:23.819 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:23 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: | | 2024-03-24T12:04:23.819 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:23 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: | | 2024-03-24T12:04:23.819 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:23 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: +----[SHA256]-----+ 2024-03-24T12:04:24.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.350+0000 7f06e2cf7700 1 Processor -- start 2024-03-24T12:04:24.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.350+0000 7f06e2cf7700 1 -- start start 2024-03-24T12:04:24.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.350+0000 7f06e2cf7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06dc1064e0 0x7f06dc1088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:24.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.350+0000 7f06e2cf7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc0796e0 con 0x7f06dc1064e0 2024-03-24T12:04:24.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.350+0000 7f06e0a93700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06dc1064e0 0x7f06dc1088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:24.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.350+0000 7f06e0a93700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06dc1064e0 0x7f06dc1088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42326/0 (socket says 172.21.15.6:42326) 2024-03-24T12:04:24.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.350+0000 7f06e0a93700 1 -- 172.21.15.6:0/2561363675 learned_addr learned my addr 172.21.15.6:0/2561363675 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:24.340 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.350+0000 7f06e0a93700 1 -- 172.21.15.6:0/2561363675 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06dc108e10 con 0x7f06dc1064e0 2024-03-24T12:04:24.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.350+0000 7f06e0a93700 1 --2- 172.21.15.6:0/2561363675 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06dc1064e0 0x7f06dc1088d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009e80 tx=0x7f06d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55e515b30e81e2e server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:24.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.351+0000 7f06db7fe700 1 -- 172.21.15.6:0/2561363675 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06d400e070 con 0x7f06dc1064e0 2024-03-24T12:04:24.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.351+0000 7f06db7fe700 1 -- 172.21.15.6:0/2561363675 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f06d4003e80 con 0x7f06dc1064e0 2024-03-24T12:04:24.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.351+0000 7f06db7fe700 1 -- 172.21.15.6:0/2561363675 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06d4004370 con 0x7f06dc1064e0 2024-03-24T12:04:24.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.351+0000 7f06e2cf7700 1 -- 172.21.15.6:0/2561363675 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06dc1064e0 msgr2=0x7f06dc1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:24.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.351+0000 7f06e2cf7700 1 --2- 172.21.15.6:0/2561363675 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06dc1064e0 0x7f06dc1088d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009e80 tx=0x7f06d400b330 comp rx=0 tx=0).stop 2024-03-24T12:04:24.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.351+0000 7f06e2cf7700 1 -- 172.21.15.6:0/2561363675 shutdown_connections 2024-03-24T12:04:24.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.351+0000 7f06e2cf7700 1 --2- 172.21.15.6:0/2561363675 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06dc1064e0 0x7f06dc1088d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:24.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.351+0000 7f06e2cf7700 1 -- 172.21.15.6:0/2561363675 >> 172.21.15.6:0/2561363675 conn(0x7f06dc0ffef0 msgr2=0x7f06dc102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:24.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.351+0000 7f06e2cf7700 1 -- 172.21.15.6:0/2561363675 shutdown_connections 2024-03-24T12:04:24.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.351+0000 7f06e2cf7700 1 -- 172.21.15.6:0/2561363675 wait complete. 2024-03-24T12:04:24.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.352+0000 7f06e2cf7700 1 Processor -- start 2024-03-24T12:04:24.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.352+0000 7f06e2cf7700 1 -- start start 2024-03-24T12:04:24.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.352+0000 7f06e2cf7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06dc1064e0 0x7f06dc199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:24.341 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.352+0000 7f06e2cf7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc199c90 con 0x7f06dc1064e0 2024-03-24T12:04:24.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.352+0000 7f06e0a93700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06dc1064e0 0x7f06dc199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:24.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.352+0000 7f06e0a93700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06dc1064e0 0x7f06dc199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42334/0 (socket says 172.21.15.6:42334) 2024-03-24T12:04:24.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.352+0000 7f06e0a93700 1 -- 172.21.15.6:0/3024216587 learned_addr learned my addr 172.21.15.6:0/3024216587 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:24.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.352+0000 7f06e0a93700 1 -- 172.21.15.6:0/3024216587 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d4009b30 con 0x7f06dc1064e0 2024-03-24T12:04:24.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.352+0000 7f06e0a93700 1 --2- 172.21.15.6:0/3024216587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06dc1064e0 0x7f06dc199750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f06d4004000 tx=0x7f06d40048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:24.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.353+0000 7f06d9ffb700 1 -- 172.21.15.6:0/3024216587 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06d400e040 con 0x7f06dc1064e0 2024-03-24T12:04:24.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.353+0000 7f06d9ffb700 1 -- 172.21.15.6:0/3024216587 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f06d400b8c0 con 0x7f06dc1064e0 2024-03-24T12:04:24.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.353+0000 7f06e2cf7700 1 -- 172.21.15.6:0/3024216587 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06dc199e90 con 0x7f06dc1064e0 2024-03-24T12:04:24.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.353+0000 7f06d9ffb700 1 -- 172.21.15.6:0/3024216587 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06d4012770 con 0x7f06dc1064e0 2024-03-24T12:04:24.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.353+0000 7f06e2cf7700 1 -- 172.21.15.6:0/3024216587 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06dc19a390 con 0x7f06dc1064e0 2024-03-24T12:04:24.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.354+0000 7f06d9ffb700 1 -- 172.21.15.6:0/3024216587 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f06d401c650 con 0x7f06dc1064e0 2024-03-24T12:04:24.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.354+0000 7f06e2cf7700 1 -- 172.21.15.6:0/3024216587 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06dc109a80 con 0x7f06dc1064e0 2024-03-24T12:04:24.342 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.354+0000 7f06d9ffb700 1 --2- 172.21.15.6:0/3024216587 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f06c4038d00 0x7f06c403b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:24.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.354+0000 7f06d9ffb700 1 -- 172.21.15.6:0/3024216587 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f06d404cd80 con 0x7f06dc1064e0 2024-03-24T12:04:24.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.354+0000 7f06dbfff700 1 --2- 172.21.15.6:0/3024216587 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f06c4038d00 0x7f06c403b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:24.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.354+0000 7f06dbfff700 1 --2- 172.21.15.6:0/3024216587 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f06c4038d00 0x7f06c403b1c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f06d0006fd0 tx=0x7f06d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:24.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.359+0000 7f06d9ffb700 1 -- 172.21.15.6:0/3024216587 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f06d401ba90 con 0x7f06dc1064e0 2024-03-24T12:04:24.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.561+0000 7f06e2cf7700 1 -- 172.21.15.6:0/3024216587 --> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f06dc1010d0 con 0x7f06c4038d00 2024-03-24T12:04:24.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.817+0000 7f06d9ffb700 1 -- 172.21.15.6:0/3024216587 <== mgr.14120 v2:172.21.15.6:6800/3010671792 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f06dc1010d0 con 0x7f06c4038d00 2024-03-24T12:04:24.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.819+0000 7f06e2cf7700 1 -- 172.21.15.6:0/3024216587 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f06c4038d00 msgr2=0x7f06c403b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:24.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.819+0000 7f06e2cf7700 1 --2- 172.21.15.6:0/3024216587 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f06c4038d00 0x7f06c403b1c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f06d0006fd0 tx=0x7f06d0006e40 comp rx=0 tx=0).stop 2024-03-24T12:04:24.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.819+0000 7f06e2cf7700 1 -- 172.21.15.6:0/3024216587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06dc1064e0 msgr2=0x7f06dc199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:24.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.819+0000 7f06e2cf7700 1 --2- 172.21.15.6:0/3024216587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06dc1064e0 0x7f06dc199750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f06d4004000 tx=0x7f06d40048d0 comp rx=0 tx=0).stop 2024-03-24T12:04:24.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.820+0000 7f06e2cf7700 1 -- 172.21.15.6:0/3024216587 shutdown_connections 2024-03-24T12:04:24.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.820+0000 7f06e2cf7700 1 --2- 172.21.15.6:0/3024216587 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f06c4038d00 0x7f06c403b1c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:24.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.820+0000 7f06e2cf7700 1 --2- 172.21.15.6:0/3024216587 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06dc1064e0 0x7f06dc199750 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:24.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.820+0000 7f06e2cf7700 1 -- 172.21.15.6:0/3024216587 >> 172.21.15.6:0/3024216587 conn(0x7f06dc0ffef0 msgr2=0x7f06dc1009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:24.343 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.820+0000 7f06e2cf7700 1 -- 172.21.15.6:0/3024216587 shutdown_connections 2024-03-24T12:04:24.344 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:23.820+0000 7f06e2cf7700 1 -- 172.21.15.6:0/3024216587 wait complete. 2024-03-24T12:04:24.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:24 smithi006 ceph-mon[30108]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:04:24.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:24 smithi006 ceph-mon[30108]: Generating ssh key... 2024-03-24T12:04:24.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:24 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' 2024-03-24T12:04:24.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:24 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' 2024-03-24T12:04:25.572 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDayS9KkD/eKNcrAuGYoDxtY9jb4r3LszONEP/OlLyGWWN9Xew86VNTH1uvp5v136RiayVgsGg7fZDbbr1JUXeoGalyoSqWMpHFWpDj0HJZk8bTepLHHpxrD/8N9CXXivK/pN/fb5ULBm8gJtdOxx9humyLOF2sKjvpzWd3X/o955Wsp2hyglbZHrSoij5pc3ly1zzN0rB32LZyWIxqMKWolTwP5MsYopVV/mrsdNreSRCapYn9Yb+15pCJrZqr8OQxrhWtn33C4/wkuk0PbzMaj+3oC2PHlHJlghz6KIUFgZ2+KLPBQJJfyay7Fo62eL/3doziecglRLjYfuvcHZfZ1dNtLJICUIWVi6RUkJJ0V42/C+WPVFJvOd2gE6P1//hCB48EqqQ+hKmlAR5Or07wOCXy8v0neuDTx+rYtrUuTKXS13dgUiWaaFgTu/1OprHlogOU3riC2GK3bp6ebAGaEtTYQnQmNYE1I42slYfDvP2zSmuBzfRRei5Rleefbm8= ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715 2024-03-24T12:04:25.572 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.797+0000 7efc5955a700 1 Processor -- start 2024-03-24T12:04:25.572 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.797+0000 7efc5955a700 1 -- start start 2024-03-24T12:04:25.572 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.797+0000 7efc5955a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc54106020 0x7efc54106440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:25.572 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.797+0000 7efc5955a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc54106a10 con 0x7efc54106020 2024-03-24T12:04:25.572 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.797+0000 7efc52ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc54106020 0x7efc54106440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:25.572 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.798+0000 7efc52ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc54106020 0x7efc54106440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42344/0 (socket says 172.21.15.6:42344) 2024-03-24T12:04:25.572 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.798+0000 7efc52ffd700 1 -- 172.21.15.6:0/4265834649 learned_addr learned my addr 172.21.15.6:0/4265834649 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:25.572 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.798+0000 7efc52ffd700 1 -- 172.21.15.6:0/4265834649 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc54104560 con 0x7efc54106020 2024-03-24T12:04:25.572 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.798+0000 7efc52ffd700 1 --2- 172.21.15.6:0/4265834649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc54106020 0x7efc54106440 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009e80 tx=0x7efc4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d0d6500a093e3b server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:25.572 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.798+0000 7efc51ffb700 1 -- 172.21.15.6:0/4265834649 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc4c00e070 con 0x7efc54106020 2024-03-24T12:04:25.573 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.798+0000 7efc51ffb700 1 -- 172.21.15.6:0/4265834649 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7efc4c003e80 con 0x7efc54106020 2024-03-24T12:04:25.573 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.798+0000 7efc51ffb700 1 -- 172.21.15.6:0/4265834649 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc4c004370 con 0x7efc54106020 2024-03-24T12:04:25.573 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.799+0000 7efc5955a700 1 -- 172.21.15.6:0/4265834649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc54106020 msgr2=0x7efc54106440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:25.573 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.799+0000 7efc5955a700 1 --2- 172.21.15.6:0/4265834649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc54106020 0x7efc54106440 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009e80 tx=0x7efc4c00b330 comp rx=0 tx=0).stop 2024-03-24T12:04:25.573 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.799+0000 7efc5955a700 1 -- 172.21.15.6:0/4265834649 shutdown_connections 2024-03-24T12:04:25.573 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.799+0000 7efc5955a700 1 --2- 172.21.15.6:0/4265834649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc54106020 0x7efc54106440 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:25.573 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.799+0000 7efc5955a700 1 -- 172.21.15.6:0/4265834649 >> 172.21.15.6:0/4265834649 conn(0x7efc540ffec0 msgr2=0x7efc54102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:25.573 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.799+0000 7efc5955a700 1 -- 172.21.15.6:0/4265834649 shutdown_connections 2024-03-24T12:04:25.573 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.799+0000 7efc5955a700 1 -- 172.21.15.6:0/4265834649 wait complete. 2024-03-24T12:04:25.573 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.799+0000 7efc5955a700 1 Processor -- start 2024-03-24T12:04:25.573 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.800+0000 7efc5955a700 1 -- start start 2024-03-24T12:04:25.573 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.800+0000 7efc5955a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc54106020 0x7efc541998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:25.573 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.800+0000 7efc5955a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc54199e00 con 0x7efc54106020 2024-03-24T12:04:25.573 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.800+0000 7efc52ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc54106020 0x7efc541998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:25.574 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.800+0000 7efc52ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc54106020 0x7efc541998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42358/0 (socket says 172.21.15.6:42358) 2024-03-24T12:04:25.574 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.800+0000 7efc52ffd700 1 -- 172.21.15.6:0/158423322 learned_addr learned my addr 172.21.15.6:0/158423322 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:25.574 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.800+0000 7efc52ffd700 1 -- 172.21.15.6:0/158423322 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4c009b30 con 0x7efc54106020 2024-03-24T12:04:25.574 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.800+0000 7efc52ffd700 1 --2- 172.21.15.6:0/158423322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc54106020 0x7efc541998c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efc4c004000 tx=0x7efc4c0044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:25.574 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.800+0000 7efc4bfff700 1 -- 172.21.15.6:0/158423322 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc4c00e040 con 0x7efc54106020 2024-03-24T12:04:25.574 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.800+0000 7efc4bfff700 1 -- 172.21.15.6:0/158423322 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7efc4c00b8c0 con 0x7efc54106020 2024-03-24T12:04:25.574 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.800+0000 7efc5955a700 1 -- 172.21.15.6:0/158423322 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5419a000 con 0x7efc54106020 2024-03-24T12:04:25.574 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.800+0000 7efc4bfff700 1 -- 172.21.15.6:0/158423322 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc4c0126c0 con 0x7efc54106020 2024-03-24T12:04:25.574 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.800+0000 7efc5955a700 1 -- 172.21.15.6:0/158423322 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5419a500 con 0x7efc54106020 2024-03-24T12:04:25.574 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.801+0000 7efc4bfff700 1 -- 172.21.15.6:0/158423322 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7efc4c012930 con 0x7efc54106020 2024-03-24T12:04:25.574 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.801+0000 7efc5955a700 1 -- 172.21.15.6:0/158423322 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc541938f0 con 0x7efc54106020 2024-03-24T12:04:25.574 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.801+0000 7efc4bfff700 1 --2- 172.21.15.6:0/158423322 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7efc40038d00 0x7efc4003b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:25.574 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.802+0000 7efc4bfff700 1 -- 172.21.15.6:0/158423322 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7efc4c04cf70 con 0x7efc54106020 2024-03-24T12:04:25.574 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.802+0000 7efc527fc700 1 --2- 172.21.15.6:0/158423322 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7efc40038d00 0x7efc4003b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:25.575 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.802+0000 7efc527fc700 1 --2- 172.21.15.6:0/158423322 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7efc40038d00 0x7efc4003b1c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efc3c006fd0 tx=0x7efc3c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:25.575 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:24.806+0000 7efc4bfff700 1 -- 172.21.15.6:0/158423322 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efc4c0119d0 con 0x7efc54106020 2024-03-24T12:04:25.575 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:25.010+0000 7efc5955a700 1 -- 172.21.15.6:0/158423322 --> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7efc540620b0 con 0x7efc40038d00 2024-03-24T12:04:25.575 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:25.010+0000 7efc4bfff700 1 -- 172.21.15.6:0/158423322 <== mgr.14120 v2:172.21.15.6:6800/3010671792 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7efc540620b0 con 0x7efc40038d00 2024-03-24T12:04:25.575 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:25.012+0000 7efc5955a700 1 -- 172.21.15.6:0/158423322 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7efc40038d00 msgr2=0x7efc4003b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:25.575 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:25.012+0000 7efc5955a700 1 --2- 172.21.15.6:0/158423322 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7efc40038d00 0x7efc4003b1c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efc3c006fd0 tx=0x7efc3c006e40 comp rx=0 tx=0).stop 2024-03-24T12:04:25.575 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:25.012+0000 7efc5955a700 1 -- 172.21.15.6:0/158423322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc54106020 msgr2=0x7efc541998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:25.575 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:25.012+0000 7efc5955a700 1 --2- 172.21.15.6:0/158423322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc54106020 0x7efc541998c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efc4c004000 tx=0x7efc4c0044e0 comp rx=0 tx=0).stop 2024-03-24T12:04:25.575 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:25.012+0000 7efc5955a700 1 -- 172.21.15.6:0/158423322 shutdown_connections 2024-03-24T12:04:25.575 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:25.012+0000 7efc5955a700 1 --2- 172.21.15.6:0/158423322 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7efc40038d00 0x7efc4003b1c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:25.575 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:25.012+0000 7efc5955a700 1 --2- 172.21.15.6:0/158423322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efc54106020 0x7efc541998c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:25.575 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:25.012+0000 7efc5955a700 1 -- 172.21.15.6:0/158423322 >> 172.21.15.6:0/158423322 conn(0x7efc540ffec0 msgr2=0x7efc54100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:25.575 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:25.013+0000 7efc5955a700 1 -- 172.21.15.6:0/158423322 shutdown_connections 2024-03-24T12:04:25.575 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:25.013+0000 7efc5955a700 1 -- 172.21.15.6:0/158423322 wait complete. 2024-03-24T12:04:25.575 INFO:teuthology.orchestra.run.smithi006.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-24T12:04:25.576 INFO:teuthology.orchestra.run.smithi006.stdout:Adding key to root@localhost authorized_keys... 2024-03-24T12:04:25.576 INFO:teuthology.orchestra.run.smithi006.stdout:Adding host smithi006... 2024-03-24T12:04:26.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:25 smithi006 ceph-mon[30108]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:04:27.109 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:26 smithi006 ceph-mon[30108]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi006", "addr": "172.21.15.6", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:04:28.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:27 smithi006 ceph-mon[30108]: Deploying cephadm binary to smithi006 2024-03-24T12:04:31.156 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Added host 'smithi006' with addr '172.21.15.6' 2024-03-24T12:04:31.156 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.044+0000 7f96092c9700 1 Processor -- start 2024-03-24T12:04:31.156 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.044+0000 7f96092c9700 1 -- start start 2024-03-24T12:04:31.156 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.044+0000 7f96092c9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9604104940 0x7f9604104d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:31.156 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.044+0000 7f96092c9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9604105330 con 0x7f9604104940 2024-03-24T12:04:31.156 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.044+0000 7f9602ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9604104940 0x7f9604104d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:31.156 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.044+0000 7f9602ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9604104940 0x7f9604104d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52362/0 (socket says 172.21.15.6:52362) 2024-03-24T12:04:31.156 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.044+0000 7f9602ffd700 1 -- 172.21.15.6:0/520023055 learned_addr learned my addr 172.21.15.6:0/520023055 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:31.156 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.044+0000 7f9602ffd700 1 -- 172.21.15.6:0/520023055 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9604105b40 con 0x7f9604104940 2024-03-24T12:04:31.156 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.045+0000 7f9602ffd700 1 --2- 172.21.15.6:0/520023055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9604104940 0x7f9604104d60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f95f40098a0 tx=0x7f95f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931ba31caa771e2d server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:31.157 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.045+0000 7f9601ffb700 1 -- 172.21.15.6:0/520023055 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95f4004160 con 0x7f9604104940 2024-03-24T12:04:31.157 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.045+0000 7f9601ffb700 1 -- 172.21.15.6:0/520023055 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f95f40042c0 con 0x7f9604104940 2024-03-24T12:04:31.157 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.045+0000 7f9601ffb700 1 -- 172.21.15.6:0/520023055 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95f40118a0 con 0x7f9604104940 2024-03-24T12:04:31.157 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.045+0000 7f96092c9700 1 -- 172.21.15.6:0/520023055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9604104940 msgr2=0x7f9604104d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:31.157 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.045+0000 7f96092c9700 1 --2- 172.21.15.6:0/520023055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9604104940 0x7f9604104d60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f95f40098a0 tx=0x7f95f4009bb0 comp rx=0 tx=0).stop 2024-03-24T12:04:31.157 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.045+0000 7f96092c9700 1 -- 172.21.15.6:0/520023055 shutdown_connections 2024-03-24T12:04:31.157 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.045+0000 7f96092c9700 1 --2- 172.21.15.6:0/520023055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9604104940 0x7f9604104d60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:31.157 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.045+0000 7f96092c9700 1 -- 172.21.15.6:0/520023055 >> 172.21.15.6:0/520023055 conn(0x7f96040fff20 msgr2=0x7f9604102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:31.157 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.045+0000 7f96092c9700 1 -- 172.21.15.6:0/520023055 shutdown_connections 2024-03-24T12:04:31.157 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.045+0000 7f96092c9700 1 -- 172.21.15.6:0/520023055 wait complete. 2024-03-24T12:04:31.157 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.046+0000 7f96092c9700 1 Processor -- start 2024-03-24T12:04:31.157 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.046+0000 7f96092c9700 1 -- start start 2024-03-24T12:04:31.157 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.046+0000 7f96092c9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9604104940 0x7f9604199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:31.157 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.046+0000 7f96092c9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9604199bc0 con 0x7f9604104940 2024-03-24T12:04:31.158 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.046+0000 7f9602ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9604104940 0x7f9604199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:31.158 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.046+0000 7f9602ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9604104940 0x7f9604199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52368/0 (socket says 172.21.15.6:52368) 2024-03-24T12:04:31.158 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.046+0000 7f9602ffd700 1 -- 172.21.15.6:0/1000871834 learned_addr learned my addr 172.21.15.6:0/1000871834 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:31.158 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.047+0000 7f9602ffd700 1 -- 172.21.15.6:0/1000871834 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f4009550 con 0x7f9604104940 2024-03-24T12:04:31.158 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.047+0000 7f9602ffd700 1 --2- 172.21.15.6:0/1000871834 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9604104940 0x7f9604199680 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f95f4003da0 tx=0x7f95f4011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:31.158 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.047+0000 7f95ebfff700 1 -- 172.21.15.6:0/1000871834 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95f4011930 con 0x7f9604104940 2024-03-24T12:04:31.158 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.047+0000 7f96092c9700 1 -- 172.21.15.6:0/1000871834 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9604199dc0 con 0x7f9604104940 2024-03-24T12:04:31.158 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.047+0000 7f95ebfff700 1 -- 172.21.15.6:0/1000871834 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f95f4003b00 con 0x7f9604104940 2024-03-24T12:04:31.158 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.047+0000 7f95ebfff700 1 -- 172.21.15.6:0/1000871834 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95f401ad50 con 0x7f9604104940 2024-03-24T12:04:31.158 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.047+0000 7f96092c9700 1 -- 172.21.15.6:0/1000871834 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960419a260 con 0x7f9604104940 2024-03-24T12:04:31.158 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.048+0000 7f95ebfff700 1 -- 172.21.15.6:0/1000871834 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f95f402d440 con 0x7f9604104940 2024-03-24T12:04:31.158 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.048+0000 7f96092c9700 1 -- 172.21.15.6:0/1000871834 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9604193810 con 0x7f9604104940 2024-03-24T12:04:31.158 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.048+0000 7f95ebfff700 1 --2- 172.21.15.6:0/1000871834 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f95ec038c60 0x7f95ec03b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:31.159 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.048+0000 7f95ebfff700 1 -- 172.21.15.6:0/1000871834 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f95f404cbc0 con 0x7f9604104940 2024-03-24T12:04:31.159 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.048+0000 7f96027fc700 1 --2- 172.21.15.6:0/1000871834 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f95ec038c60 0x7f95ec03b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:31.159 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.048+0000 7f96027fc700 1 --2- 172.21.15.6:0/1000871834 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f95ec038c60 0x7f95ec03b120 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f95f8006fd0 tx=0x7f95f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:31.159 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.053+0000 7f95ebfff700 1 -- 172.21.15.6:0/1000871834 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f95f4018a40 con 0x7f9604104940 2024-03-24T12:04:31.159 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:26.258+0000 7f96092c9700 1 -- 172.21.15.6:0/1000871834 --> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi006", "addr": "172.21.15.6", "target": ["mon-mgr", ""]}) v1 -- 0x7f960402d7d0 con 0x7f95ec038c60 2024-03-24T12:04:31.159 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:30.618+0000 7f95ebfff700 1 -- 172.21.15.6:0/1000871834 <== mgr.14120 v2:172.21.15.6:6800/3010671792 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f960402d7d0 con 0x7f95ec038c60 2024-03-24T12:04:31.159 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:30.620+0000 7f96092c9700 1 -- 172.21.15.6:0/1000871834 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f95ec038c60 msgr2=0x7f95ec03b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:31.159 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:30.620+0000 7f96092c9700 1 --2- 172.21.15.6:0/1000871834 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f95ec038c60 0x7f95ec03b120 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f95f8006fd0 tx=0x7f95f8006e40 comp rx=0 tx=0).stop 2024-03-24T12:04:31.159 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:30.620+0000 7f96092c9700 1 -- 172.21.15.6:0/1000871834 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9604104940 msgr2=0x7f9604199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:31.159 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:30.620+0000 7f96092c9700 1 --2- 172.21.15.6:0/1000871834 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9604104940 0x7f9604199680 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f95f4003da0 tx=0x7f95f4011e90 comp rx=0 tx=0).stop 2024-03-24T12:04:31.159 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:30.620+0000 7f96092c9700 1 -- 172.21.15.6:0/1000871834 shutdown_connections 2024-03-24T12:04:31.159 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:30.620+0000 7f96092c9700 1 --2- 172.21.15.6:0/1000871834 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f95ec038c60 0x7f95ec03b120 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:31.159 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:30.620+0000 7f96092c9700 1 --2- 172.21.15.6:0/1000871834 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9604104940 0x7f9604199680 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:31.160 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:30.620+0000 7f96092c9700 1 -- 172.21.15.6:0/1000871834 >> 172.21.15.6:0/1000871834 conn(0x7f96040fff20 msgr2=0x7f9604100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:31.160 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:30.621+0000 7f96092c9700 1 -- 172.21.15.6:0/1000871834 shutdown_connections 2024-03-24T12:04:31.160 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:30.621+0000 7f96092c9700 1 -- 172.21.15.6:0/1000871834 wait complete. 2024-03-24T12:04:31.160 INFO:teuthology.orchestra.run.smithi006.stdout:Deploying unmanaged mon service... 2024-03-24T12:04:31.684 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:31 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' 2024-03-24T12:04:31.685 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:31 smithi006 ceph-mon[30108]: Added host smithi006 2024-03-24T12:04:31.685 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:31 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:04:33.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:32 smithi006 ceph-mon[30108]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:04:33.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:32 smithi006 ceph-mon[30108]: Saving service mon spec with placement count:5 2024-03-24T12:04:33.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:32 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' 2024-03-24T12:04:33.321 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-24T12:04:33.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.684+0000 7faed70b8700 1 Processor -- start 2024-03-24T12:04:33.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.684+0000 7faed70b8700 1 -- start start 2024-03-24T12:04:33.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.685+0000 7faed70b8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faed0079b50 0x7faed0079f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:33.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.685+0000 7faed70b8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed007b880 con 0x7faed0079b50 2024-03-24T12:04:33.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.685+0000 7faed4e54700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faed0079b50 0x7faed0079f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:33.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.685+0000 7faed4e54700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faed0079b50 0x7faed0079f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52370/0 (socket says 172.21.15.6:52370) 2024-03-24T12:04:33.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.685+0000 7faed4e54700 1 -- 172.21.15.6:0/3924182158 learned_addr learned my addr 172.21.15.6:0/3924182158 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:33.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.685+0000 7faed4e54700 1 -- 172.21.15.6:0/3924182158 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed007b9c0 con 0x7faed0079b50 2024-03-24T12:04:33.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.685+0000 7faed4e54700 1 --2- 172.21.15.6:0/3924182158 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faed0079b50 0x7faed0079f70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faec40098a0 tx=0x7faec4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd2fbde01dfc65d server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:33.322 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.686+0000 7faecf7fe700 1 -- 172.21.15.6:0/3924182158 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faec4004160 con 0x7faed0079b50 2024-03-24T12:04:33.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.686+0000 7faecf7fe700 1 -- 172.21.15.6:0/3924182158 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7faec40042c0 con 0x7faed0079b50 2024-03-24T12:04:33.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.686+0000 7faecf7fe700 1 -- 172.21.15.6:0/3924182158 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faec40118a0 con 0x7faed0079b50 2024-03-24T12:04:33.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.686+0000 7faed70b8700 1 -- 172.21.15.6:0/3924182158 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faed0079b50 msgr2=0x7faed0079f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:33.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.686+0000 7faed70b8700 1 --2- 172.21.15.6:0/3924182158 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faed0079b50 0x7faed0079f70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faec40098a0 tx=0x7faec4009bb0 comp rx=0 tx=0).stop 2024-03-24T12:04:33.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.686+0000 7faed70b8700 1 -- 172.21.15.6:0/3924182158 shutdown_connections 2024-03-24T12:04:33.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.686+0000 7faed70b8700 1 --2- 172.21.15.6:0/3924182158 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faed0079b50 0x7faed0079f70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:33.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.686+0000 7faed70b8700 1 -- 172.21.15.6:0/3924182158 >> 172.21.15.6:0/3924182158 conn(0x7faed0101000 msgr2=0x7faed0103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:33.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.686+0000 7faed70b8700 1 -- 172.21.15.6:0/3924182158 shutdown_connections 2024-03-24T12:04:33.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.686+0000 7faed70b8700 1 -- 172.21.15.6:0/3924182158 wait complete. 2024-03-24T12:04:33.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.687+0000 7faed70b8700 1 Processor -- start 2024-03-24T12:04:33.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.687+0000 7faed70b8700 1 -- start start 2024-03-24T12:04:33.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.687+0000 7faed70b8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faed0079b50 0x7faed019b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:33.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.687+0000 7faed70b8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed019be70 con 0x7faed0079b50 2024-03-24T12:04:33.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.687+0000 7faed4e54700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faed0079b50 0x7faed019b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:33.323 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.687+0000 7faed4e54700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faed0079b50 0x7faed019b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52386/0 (socket says 172.21.15.6:52386) 2024-03-24T12:04:33.324 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.687+0000 7faed4e54700 1 -- 172.21.15.6:0/3346538740 learned_addr learned my addr 172.21.15.6:0/3346538740 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:33.324 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.687+0000 7faed4e54700 1 -- 172.21.15.6:0/3346538740 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec4009550 con 0x7faed0079b50 2024-03-24T12:04:33.324 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.688+0000 7faed4e54700 1 --2- 172.21.15.6:0/3346538740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faed0079b50 0x7faed019b930 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faec4003da0 tx=0x7faec4011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:33.324 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.688+0000 7faecdffb700 1 -- 172.21.15.6:0/3346538740 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faec4011930 con 0x7faed0079b50 2024-03-24T12:04:33.324 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.688+0000 7faecdffb700 1 -- 172.21.15.6:0/3346538740 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7faec4003b00 con 0x7faed0079b50 2024-03-24T12:04:33.324 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.688+0000 7faed70b8700 1 -- 172.21.15.6:0/3346538740 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed019c070 con 0x7faed0079b50 2024-03-24T12:04:33.324 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.688+0000 7faecdffb700 1 -- 172.21.15.6:0/3346538740 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faec401ad50 con 0x7faed0079b50 2024-03-24T12:04:33.324 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.688+0000 7faed70b8700 1 -- 172.21.15.6:0/3346538740 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed019c510 con 0x7faed0079b50 2024-03-24T12:04:33.324 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.689+0000 7faecdffb700 1 -- 172.21.15.6:0/3346538740 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7faec40036a0 con 0x7faed0079b50 2024-03-24T12:04:33.324 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.689+0000 7faecdffb700 1 --2- 172.21.15.6:0/3346538740 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7faec0038cb0 0x7faec003b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:33.324 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.689+0000 7faed70b8700 1 -- 172.21.15.6:0/3346538740 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed010bdd0 con 0x7faed0079b50 2024-03-24T12:04:33.324 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.689+0000 7faecdffb700 1 -- 172.21.15.6:0/3346538740 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7faec404cb30 con 0x7faed0079b50 2024-03-24T12:04:33.324 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.689+0000 7faecffff700 1 --2- 172.21.15.6:0/3346538740 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7faec0038cb0 0x7faec003b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:33.324 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.690+0000 7faecffff700 1 --2- 172.21.15.6:0/3346538740 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7faec0038cb0 0x7faec003b170 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faebc006fd0 tx=0x7faebc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:33.325 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.694+0000 7faecdffb700 1 -- 172.21.15.6:0/3346538740 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faec40238a0 con 0x7faed0079b50 2024-03-24T12:04:33.325 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.920+0000 7faed70b8700 1 -- 172.21.15.6:0/3346538740 --> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7faed007b9c0 con 0x7faec0038cb0 2024-03-24T12:04:33.325 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.927+0000 7faecdffb700 1 -- 172.21.15.6:0/3346538740 <== mgr.14120 v2:172.21.15.6:6800/3010671792 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7faed007b9c0 con 0x7faec0038cb0 2024-03-24T12:04:33.325 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.929+0000 7faed70b8700 1 -- 172.21.15.6:0/3346538740 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7faec0038cb0 msgr2=0x7faec003b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:33.325 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.929+0000 7faed70b8700 1 --2- 172.21.15.6:0/3346538740 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7faec0038cb0 0x7faec003b170 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faebc006fd0 tx=0x7faebc006e40 comp rx=0 tx=0).stop 2024-03-24T12:04:33.325 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.929+0000 7faed70b8700 1 -- 172.21.15.6:0/3346538740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faed0079b50 msgr2=0x7faed019b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:33.325 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.929+0000 7faed70b8700 1 --2- 172.21.15.6:0/3346538740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faed0079b50 0x7faed019b930 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faec4003da0 tx=0x7faec4011e90 comp rx=0 tx=0).stop 2024-03-24T12:04:33.325 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.929+0000 7faed70b8700 1 -- 172.21.15.6:0/3346538740 shutdown_connections 2024-03-24T12:04:33.325 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.929+0000 7faed70b8700 1 --2- 172.21.15.6:0/3346538740 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7faec0038cb0 0x7faec003b170 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:33.325 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.929+0000 7faed70b8700 1 --2- 172.21.15.6:0/3346538740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faed0079b50 0x7faed019b930 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:33.325 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.929+0000 7faed70b8700 1 -- 172.21.15.6:0/3346538740 >> 172.21.15.6:0/3346538740 conn(0x7faed0101000 msgr2=0x7faed0101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:33.325 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.930+0000 7faed70b8700 1 -- 172.21.15.6:0/3346538740 shutdown_connections 2024-03-24T12:04:33.325 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:31.930+0000 7faed70b8700 1 -- 172.21.15.6:0/3346538740 wait complete. 2024-03-24T12:04:33.325 INFO:teuthology.orchestra.run.smithi006.stdout:Deploying unmanaged mgr service... 2024-03-24T12:04:34.307 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:33 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' 2024-03-24T12:04:34.307 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:33 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' 2024-03-24T12:04:34.562 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-24T12:04:34.562 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.772+0000 7f51626bf700 1 Processor -- start 2024-03-24T12:04:34.562 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.772+0000 7f51626bf700 1 -- start start 2024-03-24T12:04:34.562 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.773+0000 7f51626bf700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f515c104900 0x7f515c104d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:34.562 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.773+0000 7f51626bf700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515c1052f0 con 0x7f515c104900 2024-03-24T12:04:34.562 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.773+0000 7f515bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f515c104900 0x7f515c104d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:34.562 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.773+0000 7f515bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f515c104900 0x7f515c104d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52392/0 (socket says 172.21.15.6:52392) 2024-03-24T12:04:34.562 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.773+0000 7f515bfff700 1 -- 172.21.15.6:0/4208801803 learned_addr learned my addr 172.21.15.6:0/4208801803 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:34.562 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.773+0000 7f515bfff700 1 -- 172.21.15.6:0/4208801803 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515c105b00 con 0x7f515c104900 2024-03-24T12:04:34.563 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.773+0000 7f515bfff700 1 --2- 172.21.15.6:0/4208801803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f515c104900 0x7f515c104d20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f514c0098a0 tx=0x7f514c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea5cdfbf4564267 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:34.563 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.773+0000 7f515affd700 1 -- 172.21.15.6:0/4208801803 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f514c004160 con 0x7f515c104900 2024-03-24T12:04:34.563 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.774+0000 7f515affd700 1 -- 172.21.15.6:0/4208801803 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f514c0042c0 con 0x7f515c104900 2024-03-24T12:04:34.563 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.774+0000 7f515affd700 1 -- 172.21.15.6:0/4208801803 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f514c011850 con 0x7f515c104900 2024-03-24T12:04:34.563 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.774+0000 7f51626bf700 1 -- 172.21.15.6:0/4208801803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f515c104900 msgr2=0x7f515c104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:34.563 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.774+0000 7f51626bf700 1 --2- 172.21.15.6:0/4208801803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f515c104900 0x7f515c104d20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f514c0098a0 tx=0x7f514c009bb0 comp rx=0 tx=0).stop 2024-03-24T12:04:34.563 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.774+0000 7f51626bf700 1 -- 172.21.15.6:0/4208801803 shutdown_connections 2024-03-24T12:04:34.563 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.774+0000 7f51626bf700 1 --2- 172.21.15.6:0/4208801803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f515c104900 0x7f515c104d20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:34.563 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.774+0000 7f51626bf700 1 -- 172.21.15.6:0/4208801803 >> 172.21.15.6:0/4208801803 conn(0x7f515c0ffec0 msgr2=0x7f515c1022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:34.563 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.774+0000 7f51626bf700 1 -- 172.21.15.6:0/4208801803 shutdown_connections 2024-03-24T12:04:34.563 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.775+0000 7f51626bf700 1 -- 172.21.15.6:0/4208801803 wait complete. 2024-03-24T12:04:34.563 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.775+0000 7f51626bf700 1 Processor -- start 2024-03-24T12:04:34.563 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.775+0000 7f51626bf700 1 -- start start 2024-03-24T12:04:34.563 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.776+0000 7f51626bf700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f515c104900 0x7f515c078a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:34.564 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.776+0000 7f51626bf700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515c078f70 con 0x7f515c104900 2024-03-24T12:04:34.564 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.776+0000 7f515bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f515c104900 0x7f515c078a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:34.564 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.776+0000 7f515bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f515c104900 0x7f515c078a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52394/0 (socket says 172.21.15.6:52394) 2024-03-24T12:04:34.564 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.776+0000 7f515bfff700 1 -- 172.21.15.6:0/164866473 learned_addr learned my addr 172.21.15.6:0/164866473 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:34.564 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.776+0000 7f515bfff700 1 -- 172.21.15.6:0/164866473 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c009550 con 0x7f515c104900 2024-03-24T12:04:34.564 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.776+0000 7f515bfff700 1 --2- 172.21.15.6:0/164866473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f515c104900 0x7f515c078a30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f514c000c00 tx=0x7f514c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:34.564 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.776+0000 7f51597fa700 1 -- 172.21.15.6:0/164866473 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f514c0036a0 con 0x7f515c104900 2024-03-24T12:04:34.564 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.776+0000 7f51597fa700 1 -- 172.21.15.6:0/164866473 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f514c024440 con 0x7f515c104900 2024-03-24T12:04:34.564 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.776+0000 7f51597fa700 1 -- 172.21.15.6:0/164866473 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f514c01a430 con 0x7f515c104900 2024-03-24T12:04:34.564 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.776+0000 7f51626bf700 1 -- 172.21.15.6:0/164866473 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515c19c450 con 0x7f515c104900 2024-03-24T12:04:34.564 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.777+0000 7f51626bf700 1 -- 172.21.15.6:0/164866473 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f515c19c8f0 con 0x7f515c104900 2024-03-24T12:04:34.564 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.777+0000 7f51626bf700 1 -- 172.21.15.6:0/164866473 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f515c0632a0 con 0x7f515c104900 2024-03-24T12:04:34.564 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.778+0000 7f51597fa700 1 -- 172.21.15.6:0/164866473 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f514c01a8b0 con 0x7f515c104900 2024-03-24T12:04:34.564 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.778+0000 7f51597fa700 1 --2- 172.21.15.6:0/164866473 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f5144038d00 0x7f514403b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:34.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.778+0000 7f51597fa700 1 -- 172.21.15.6:0/164866473 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f514c04cd50 con 0x7f515c104900 2024-03-24T12:04:34.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.778+0000 7f515b7fe700 1 --2- 172.21.15.6:0/164866473 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f5144038d00 0x7f514403b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:34.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.778+0000 7f515b7fe700 1 --2- 172.21.15.6:0/164866473 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f5144038d00 0x7f514403b1c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5150006fd0 tx=0x7f5150006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:34.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:33.783+0000 7f51597fa700 1 -- 172.21.15.6:0/164866473 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f514c018dc0 con 0x7f515c104900 2024-03-24T12:04:34.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:34.016+0000 7f51626bf700 1 -- 172.21.15.6:0/164866473 --> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f515c1011d0 con 0x7f5144038d00 2024-03-24T12:04:34.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:34.021+0000 7f51597fa700 1 -- 172.21.15.6:0/164866473 <== mgr.14120 v2:172.21.15.6:6800/3010671792 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f515c1011d0 con 0x7f5144038d00 2024-03-24T12:04:34.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:34.023+0000 7f51626bf700 1 -- 172.21.15.6:0/164866473 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f5144038d00 msgr2=0x7f514403b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:34.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:34.023+0000 7f51626bf700 1 --2- 172.21.15.6:0/164866473 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f5144038d00 0x7f514403b1c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5150006fd0 tx=0x7f5150006e40 comp rx=0 tx=0).stop 2024-03-24T12:04:34.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:34.023+0000 7f51626bf700 1 -- 172.21.15.6:0/164866473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f515c104900 msgr2=0x7f515c078a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:34.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:34.023+0000 7f51626bf700 1 --2- 172.21.15.6:0/164866473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f515c104900 0x7f515c078a30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f514c000c00 tx=0x7f514c011ac0 comp rx=0 tx=0).stop 2024-03-24T12:04:34.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:34.023+0000 7f51626bf700 1 -- 172.21.15.6:0/164866473 shutdown_connections 2024-03-24T12:04:34.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:34.023+0000 7f51626bf700 1 --2- 172.21.15.6:0/164866473 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f5144038d00 0x7f514403b1c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:34.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:34.023+0000 7f51626bf700 1 --2- 172.21.15.6:0/164866473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f515c104900 0x7f515c078a30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:34.565 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:34.023+0000 7f51626bf700 1 -- 172.21.15.6:0/164866473 >> 172.21.15.6:0/164866473 conn(0x7f515c0ffec0 msgr2=0x7f515c100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:34.566 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:34.023+0000 7f51626bf700 1 -- 172.21.15.6:0/164866473 shutdown_connections 2024-03-24T12:04:34.566 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:34.023+0000 7f51626bf700 1 -- 172.21.15.6:0/164866473 wait complete. 2024-03-24T12:04:35.287 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:35 smithi006 ceph-mon[30108]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:04:35.287 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:35 smithi006 ceph-mon[30108]: Saving service mgr spec with placement count:2 2024-03-24T12:04:35.287 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:35 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' 2024-03-24T12:04:36.531 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.662+0000 7f74f5ff7700 1 Processor -- start 2024-03-24T12:04:36.531 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.662+0000 7f74f5ff7700 1 -- start start 2024-03-24T12:04:36.531 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.662+0000 7f74f5ff7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f0104e00 0x7f74f0105220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:36.532 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.662+0000 7f74f5ff7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f01057f0 con 0x7f74f0104e00 2024-03-24T12:04:36.532 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.662+0000 7f74ef7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f0104e00 0x7f74f0105220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:36.532 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.662+0000 7f74ef7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f0104e00 0x7f74f0105220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51890/0 (socket says 172.21.15.6:51890) 2024-03-24T12:04:36.532 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.662+0000 7f74ef7fe700 1 -- 172.21.15.6:0/42551162 learned_addr learned my addr 172.21.15.6:0/42551162 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:36.532 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.663+0000 7f74ef7fe700 1 -- 172.21.15.6:0/42551162 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f0105b20 con 0x7f74f0104e00 2024-03-24T12:04:36.532 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.663+0000 7f74ef7fe700 1 --2- 172.21.15.6:0/42551162 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f0104e00 0x7f74f0105220 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f74e00098a0 tx=0x7f74e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991230038dd61411 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:36.532 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.663+0000 7f74ee7fc700 1 -- 172.21.15.6:0/42551162 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74e0004160 con 0x7f74f0104e00 2024-03-24T12:04:36.532 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.663+0000 7f74ee7fc700 1 -- 172.21.15.6:0/42551162 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f74e00042c0 con 0x7f74f0104e00 2024-03-24T12:04:36.532 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.663+0000 7f74ee7fc700 1 -- 172.21.15.6:0/42551162 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74e0011850 con 0x7f74f0104e00 2024-03-24T12:04:36.532 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.664+0000 7f74f5ff7700 1 -- 172.21.15.6:0/42551162 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f0104e00 msgr2=0x7f74f0105220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:36.532 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.664+0000 7f74f5ff7700 1 --2- 172.21.15.6:0/42551162 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f0104e00 0x7f74f0105220 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f74e00098a0 tx=0x7f74e0009bb0 comp rx=0 tx=0).stop 2024-03-24T12:04:36.532 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.664+0000 7f74f5ff7700 1 -- 172.21.15.6:0/42551162 shutdown_connections 2024-03-24T12:04:36.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.664+0000 7f74f5ff7700 1 --2- 172.21.15.6:0/42551162 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f0104e00 0x7f74f0105220 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:36.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.664+0000 7f74f5ff7700 1 -- 172.21.15.6:0/42551162 >> 172.21.15.6:0/42551162 conn(0x7f74f00ff640 msgr2=0x7f74f0101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:36.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.664+0000 7f74f5ff7700 1 -- 172.21.15.6:0/42551162 shutdown_connections 2024-03-24T12:04:36.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.664+0000 7f74f5ff7700 1 -- 172.21.15.6:0/42551162 wait complete. 2024-03-24T12:04:36.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.664+0000 7f74f5ff7700 1 Processor -- start 2024-03-24T12:04:36.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.665+0000 7f74f5ff7700 1 -- start start 2024-03-24T12:04:36.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.665+0000 7f74f5ff7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f0104e00 0x7f74f018f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:36.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.665+0000 7f74f5ff7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f018fe00 con 0x7f74f0104e00 2024-03-24T12:04:36.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.665+0000 7f74ef7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f0104e00 0x7f74f018f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:36.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.665+0000 7f74ef7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f0104e00 0x7f74f018f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51894/0 (socket says 172.21.15.6:51894) 2024-03-24T12:04:36.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.665+0000 7f74ef7fe700 1 -- 172.21.15.6:0/2016811685 learned_addr learned my addr 172.21.15.6:0/2016811685 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:36.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.665+0000 7f74ef7fe700 1 -- 172.21.15.6:0/2016811685 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e0009550 con 0x7f74f0104e00 2024-03-24T12:04:36.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.665+0000 7f74ef7fe700 1 --2- 172.21.15.6:0/2016811685 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f0104e00 0x7f74f018f8c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f74e0006af0 tx=0x7f74e0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:36.533 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.665+0000 7f74ecff9700 1 -- 172.21.15.6:0/2016811685 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74e00036a0 con 0x7f74f0104e00 2024-03-24T12:04:36.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.665+0000 7f74ecff9700 1 -- 172.21.15.6:0/2016811685 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f74e001a430 con 0x7f74f0104e00 2024-03-24T12:04:36.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.666+0000 7f74ecff9700 1 -- 172.21.15.6:0/2016811685 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74e001ae60 con 0x7f74f0104e00 2024-03-24T12:04:36.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.666+0000 7f74f5ff7700 1 -- 172.21.15.6:0/2016811685 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f018dff0 con 0x7f74f0104e00 2024-03-24T12:04:36.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.666+0000 7f74f5ff7700 1 -- 172.21.15.6:0/2016811685 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f018e490 con 0x7f74f0104e00 2024-03-24T12:04:36.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.666+0000 7f74ecff9700 1 -- 172.21.15.6:0/2016811685 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f74e0019450 con 0x7f74f0104e00 2024-03-24T12:04:36.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.666+0000 7f74f5ff7700 1 -- 172.21.15.6:0/2016811685 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f00632a0 con 0x7f74f0104e00 2024-03-24T12:04:36.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.667+0000 7f74ecff9700 1 --2- 172.21.15.6:0/2016811685 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f74d8038cb0 0x7f74d803b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:36.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.667+0000 7f74ecff9700 1 -- 172.21.15.6:0/2016811685 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f74e004ce50 con 0x7f74f0104e00 2024-03-24T12:04:36.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.667+0000 7f74eeffd700 1 --2- 172.21.15.6:0/2016811685 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f74d8038cb0 0x7f74d803b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:36.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.667+0000 7f74eeffd700 1 --2- 172.21.15.6:0/2016811685 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f74d8038cb0 0x7f74d803b170 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f74e4006fd0 tx=0x7f74e4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:36.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.672+0000 7f74ecff9700 1 -- 172.21.15.6:0/2016811685 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f74e0018c70 con 0x7f74f0104e00 2024-03-24T12:04:36.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.872+0000 7f74f5ff7700 1 -- 172.21.15.6:0/2016811685 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f74f0108d50 con 0x7f74f0104e00 2024-03-24T12:04:36.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.877+0000 7f74ecff9700 1 -- 172.21.15.6:0/2016811685 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f74e0018860 con 0x7f74f0104e00 2024-03-24T12:04:36.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.879+0000 7f74f5ff7700 1 -- 172.21.15.6:0/2016811685 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f74d8038cb0 msgr2=0x7f74d803b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:36.534 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.879+0000 7f74f5ff7700 1 --2- 172.21.15.6:0/2016811685 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f74d8038cb0 0x7f74d803b170 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f74e4006fd0 tx=0x7f74e4006e40 comp rx=0 tx=0).stop 2024-03-24T12:04:36.535 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.879+0000 7f74f5ff7700 1 -- 172.21.15.6:0/2016811685 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f0104e00 msgr2=0x7f74f018f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:36.535 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.879+0000 7f74f5ff7700 1 --2- 172.21.15.6:0/2016811685 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f0104e00 0x7f74f018f8c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f74e0006af0 tx=0x7f74e0011a30 comp rx=0 tx=0).stop 2024-03-24T12:04:36.535 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.879+0000 7f74f5ff7700 1 -- 172.21.15.6:0/2016811685 shutdown_connections 2024-03-24T12:04:36.535 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.879+0000 7f74f5ff7700 1 --2- 172.21.15.6:0/2016811685 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f74d8038cb0 0x7f74d803b170 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:36.535 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.879+0000 7f74f5ff7700 1 --2- 172.21.15.6:0/2016811685 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74f0104e00 0x7f74f018f8c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:36.535 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.879+0000 7f74f5ff7700 1 -- 172.21.15.6:0/2016811685 >> 172.21.15.6:0/2016811685 conn(0x7f74f00ff640 msgr2=0x7f74f0100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:36.535 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.879+0000 7f74f5ff7700 1 -- 172.21.15.6:0/2016811685 shutdown_connections 2024-03-24T12:04:36.535 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:35.879+0000 7f74f5ff7700 1 -- 172.21.15.6:0/2016811685 wait complete. 2024-03-24T12:04:36.774 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:36 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' 2024-03-24T12:04:36.774 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:36 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2016811685' entity='client.admin' 2024-03-24T12:04:37.812 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.994+0000 7f1bf79a9700 1 Processor -- start 2024-03-24T12:04:37.812 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.995+0000 7f1bf79a9700 1 -- start start 2024-03-24T12:04:37.813 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.995+0000 7f1bf79a9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bf01042d0 0x7f1bf01046f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:37.813 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.995+0000 7f1bf79a9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf00796e0 con 0x7f1bf01042d0 2024-03-24T12:04:37.813 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.995+0000 7f1bf5745700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bf01042d0 0x7f1bf01046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:37.813 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.995+0000 7f1bf5745700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bf01042d0 0x7f1bf01046f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51904/0 (socket says 172.21.15.6:51904) 2024-03-24T12:04:37.813 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.995+0000 7f1bf5745700 1 -- 172.21.15.6:0/1179005497 learned_addr learned my addr 172.21.15.6:0/1179005497 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:37.813 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.995+0000 7f1bf5745700 1 -- 172.21.15.6:0/1179005497 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf0104c30 con 0x7f1bf01042d0 2024-03-24T12:04:37.813 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.995+0000 7f1bf5745700 1 --2- 172.21.15.6:0/1179005497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bf01042d0 0x7f1bf01046f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0098a0 tx=0x7f1bec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b0294e02109768 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:37.813 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.995+0000 7f1be7fff700 1 -- 172.21.15.6:0/1179005497 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bec004160 con 0x7f1bf01042d0 2024-03-24T12:04:37.813 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.995+0000 7f1be7fff700 1 -- 172.21.15.6:0/1179005497 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1bec0042c0 con 0x7f1bf01042d0 2024-03-24T12:04:37.813 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.995+0000 7f1be7fff700 1 -- 172.21.15.6:0/1179005497 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bec011850 con 0x7f1bf01042d0 2024-03-24T12:04:37.813 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.996+0000 7f1bf79a9700 1 -- 172.21.15.6:0/1179005497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bf01042d0 msgr2=0x7f1bf01046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:37.813 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.996+0000 7f1bf79a9700 1 --2- 172.21.15.6:0/1179005497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bf01042d0 0x7f1bf01046f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0098a0 tx=0x7f1bec009bb0 comp rx=0 tx=0).stop 2024-03-24T12:04:37.813 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.996+0000 7f1bf79a9700 1 -- 172.21.15.6:0/1179005497 shutdown_connections 2024-03-24T12:04:37.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.996+0000 7f1bf79a9700 1 --2- 172.21.15.6:0/1179005497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bf01042d0 0x7f1bf01046f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:37.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.996+0000 7f1bf79a9700 1 -- 172.21.15.6:0/1179005497 >> 172.21.15.6:0/1179005497 conn(0x7f1bf00ffef0 msgr2=0x7f1bf0102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:37.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.996+0000 7f1bf79a9700 1 -- 172.21.15.6:0/1179005497 shutdown_connections 2024-03-24T12:04:37.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.996+0000 7f1bf79a9700 1 -- 172.21.15.6:0/1179005497 wait complete. 2024-03-24T12:04:37.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.997+0000 7f1bf79a9700 1 Processor -- start 2024-03-24T12:04:37.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.997+0000 7f1bf79a9700 1 -- start start 2024-03-24T12:04:37.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.997+0000 7f1bf79a9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bf01042d0 0x7f1bf019b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:37.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.997+0000 7f1bf79a9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf019bdc0 con 0x7f1bf01042d0 2024-03-24T12:04:37.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.997+0000 7f1bf5745700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bf01042d0 0x7f1bf019b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:37.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.997+0000 7f1bf5745700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bf01042d0 0x7f1bf019b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51920/0 (socket says 172.21.15.6:51920) 2024-03-24T12:04:37.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.997+0000 7f1bf5745700 1 -- 172.21.15.6:0/153996578 learned_addr learned my addr 172.21.15.6:0/153996578 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:37.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.997+0000 7f1bf5745700 1 -- 172.21.15.6:0/153996578 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bec009550 con 0x7f1bf01042d0 2024-03-24T12:04:37.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.997+0000 7f1bf5745700 1 --2- 172.21.15.6:0/153996578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bf01042d0 0x7f1bf019b880 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1bec006af0 tx=0x7f1bec011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:37.814 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.998+0000 7f1be67fc700 1 -- 172.21.15.6:0/153996578 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bec0036a0 con 0x7f1bf01042d0 2024-03-24T12:04:37.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.998+0000 7f1be67fc700 1 -- 172.21.15.6:0/153996578 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1bec01a430 con 0x7f1bf01042d0 2024-03-24T12:04:37.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.998+0000 7f1bf79a9700 1 -- 172.21.15.6:0/153996578 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bf019bfc0 con 0x7f1bf01042d0 2024-03-24T12:04:37.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.998+0000 7f1be67fc700 1 -- 172.21.15.6:0/153996578 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bec0193c0 con 0x7f1bf01042d0 2024-03-24T12:04:37.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.998+0000 7f1bf79a9700 1 -- 172.21.15.6:0/153996578 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bf019c460 con 0x7f1bf01042d0 2024-03-24T12:04:37.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.998+0000 7f1be67fc700 1 -- 172.21.15.6:0/153996578 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f1bec019680 con 0x7f1bf01042d0 2024-03-24T12:04:37.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.999+0000 7f1bf79a9700 1 -- 172.21.15.6:0/153996578 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bf0195550 con 0x7f1bf01042d0 2024-03-24T12:04:37.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.999+0000 7f1be67fc700 1 --2- 172.21.15.6:0/153996578 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f1bdc038c60 0x7f1bdc03b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:37.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.999+0000 7f1be67fc700 1 -- 172.21.15.6:0/153996578 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1bec04c9f0 con 0x7f1bf01042d0 2024-03-24T12:04:37.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.999+0000 7f1bf4f44700 1 --2- 172.21.15.6:0/153996578 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f1bdc038c60 0x7f1bdc03b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:37.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:36.999+0000 7f1bf4f44700 1 --2- 172.21.15.6:0/153996578 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f1bdc038c60 0x7f1bdc03b120 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1be0006fd0 tx=0x7f1be0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:37.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:37.004+0000 7f1be67fc700 1 -- 172.21.15.6:0/153996578 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1bec018350 con 0x7f1bf01042d0 2024-03-24T12:04:37.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:37.224+0000 7f1bf79a9700 1 -- 172.21.15.6:0/153996578 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f1bf0105a20 con 0x7f1bf01042d0 2024-03-24T12:04:37.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:37.227+0000 7f1be67fc700 1 -- 172.21.15.6:0/153996578 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f1bec023c20 con 0x7f1bf01042d0 2024-03-24T12:04:37.815 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:37.229+0000 7f1bf79a9700 1 -- 172.21.15.6:0/153996578 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f1bdc038c60 msgr2=0x7f1bdc03b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:37.816 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:37.229+0000 7f1bf79a9700 1 --2- 172.21.15.6:0/153996578 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f1bdc038c60 0x7f1bdc03b120 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1be0006fd0 tx=0x7f1be0006e40 comp rx=0 tx=0).stop 2024-03-24T12:04:37.816 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:37.229+0000 7f1bf79a9700 1 -- 172.21.15.6:0/153996578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bf01042d0 msgr2=0x7f1bf019b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:37.816 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:37.229+0000 7f1bf79a9700 1 --2- 172.21.15.6:0/153996578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bf01042d0 0x7f1bf019b880 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1bec006af0 tx=0x7f1bec011a50 comp rx=0 tx=0).stop 2024-03-24T12:04:37.816 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:37.230+0000 7f1bf79a9700 1 -- 172.21.15.6:0/153996578 shutdown_connections 2024-03-24T12:04:37.816 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:37.230+0000 7f1bf79a9700 1 --2- 172.21.15.6:0/153996578 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f1bdc038c60 0x7f1bdc03b120 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:37.816 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:37.230+0000 7f1bf79a9700 1 --2- 172.21.15.6:0/153996578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bf01042d0 0x7f1bf019b880 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:37.816 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:37.230+0000 7f1bf79a9700 1 -- 172.21.15.6:0/153996578 >> 172.21.15.6:0/153996578 conn(0x7f1bf00ffef0 msgr2=0x7f1bf0192690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:37.816 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:37.230+0000 7f1bf79a9700 1 -- 172.21.15.6:0/153996578 shutdown_connections 2024-03-24T12:04:37.816 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:37.230+0000 7f1bf79a9700 1 -- 172.21.15.6:0/153996578 wait complete. 2024-03-24T12:04:37.816 INFO:teuthology.orchestra.run.smithi006.stdout:Enabling the dashboard module... 2024-03-24T12:04:38.107 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:37 smithi006 ceph-mon[30108]: from='mgr.14120 172.21.15.6:0/3662447871' entity='mgr.y' 2024-03-24T12:04:38.107 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/153996578' entity='client.admin' 2024-03-24T12:04:39.123 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:38 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/219685784' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-24T12:04:39.123 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:38 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ignoring --setuser ceph since I am not root 2024-03-24T12:04:39.123 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:38 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ignoring --setgroup ceph since I am not root 2024-03-24T12:04:39.398 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.214+0000 7fc7a56c0700 1 Processor -- start 2024-03-24T12:04:39.398 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.214+0000 7fc7a56c0700 1 -- start start 2024-03-24T12:04:39.398 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.215+0000 7fc7a56c0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7a01064c0 0x7fc7a01088b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:39.398 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.215+0000 7fc7a56c0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a00796c0 con 0x7fc7a01064c0 2024-03-24T12:04:39.399 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.215+0000 7fc79effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7a01064c0 0x7fc7a01088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:39.399 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.215+0000 7fc79effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7a01064c0 0x7fc7a01088b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51922/0 (socket says 172.21.15.6:51922) 2024-03-24T12:04:39.399 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.215+0000 7fc79effd700 1 -- 172.21.15.6:0/4031610570 learned_addr learned my addr 172.21.15.6:0/4031610570 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:39.399 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.215+0000 7fc79effd700 1 -- 172.21.15.6:0/4031610570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a0108df0 con 0x7fc7a01064c0 2024-03-24T12:04:39.399 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.215+0000 7fc79effd700 1 --2- 172.21.15.6:0/4031610570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7a01064c0 0x7fc7a01088b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc7900098a0 tx=0x7fc790009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3feec01fa89b63 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:39.399 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.216+0000 7fc79dffb700 1 -- 172.21.15.6:0/4031610570 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc790004160 con 0x7fc7a01064c0 2024-03-24T12:04:39.399 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.216+0000 7fc79dffb700 1 -- 172.21.15.6:0/4031610570 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fc7900042c0 con 0x7fc7a01064c0 2024-03-24T12:04:39.399 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.216+0000 7fc79dffb700 1 -- 172.21.15.6:0/4031610570 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc790011850 con 0x7fc7a01064c0 2024-03-24T12:04:39.399 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.216+0000 7fc7a56c0700 1 -- 172.21.15.6:0/4031610570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7a01064c0 msgr2=0x7fc7a01088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:39.399 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.216+0000 7fc7a56c0700 1 --2- 172.21.15.6:0/4031610570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7a01064c0 0x7fc7a01088b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc7900098a0 tx=0x7fc790009bb0 comp rx=0 tx=0).stop 2024-03-24T12:04:39.399 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.216+0000 7fc7a56c0700 1 -- 172.21.15.6:0/4031610570 shutdown_connections 2024-03-24T12:04:39.399 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.216+0000 7fc7a56c0700 1 --2- 172.21.15.6:0/4031610570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7a01064c0 0x7fc7a01088b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:39.399 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.216+0000 7fc7a56c0700 1 -- 172.21.15.6:0/4031610570 >> 172.21.15.6:0/4031610570 conn(0x7fc7a00ffe50 msgr2=0x7fc7a01022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:39.399 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.216+0000 7fc7a56c0700 1 -- 172.21.15.6:0/4031610570 shutdown_connections 2024-03-24T12:04:39.400 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.217+0000 7fc7a56c0700 1 -- 172.21.15.6:0/4031610570 wait complete. 2024-03-24T12:04:39.400 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.217+0000 7fc7a56c0700 1 Processor -- start 2024-03-24T12:04:39.400 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.217+0000 7fc7a56c0700 1 -- start start 2024-03-24T12:04:39.400 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.218+0000 7fc7a56c0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7a01064c0 0x7fc7a0105d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:39.400 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.218+0000 7fc7a56c0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a0106280 con 0x7fc7a01064c0 2024-03-24T12:04:39.400 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.218+0000 7fc79effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7a01064c0 0x7fc7a0105d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:39.400 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.218+0000 7fc79effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7a01064c0 0x7fc7a0105d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51936/0 (socket says 172.21.15.6:51936) 2024-03-24T12:04:39.400 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.218+0000 7fc79effd700 1 -- 172.21.15.6:0/219685784 learned_addr learned my addr 172.21.15.6:0/219685784 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:39.400 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.218+0000 7fc79effd700 1 -- 172.21.15.6:0/219685784 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc790009550 con 0x7fc7a01064c0 2024-03-24T12:04:39.400 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.218+0000 7fc79effd700 1 --2- 172.21.15.6:0/219685784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7a01064c0 0x7fc7a0105d40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc790006af0 tx=0x7fc790011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:39.400 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.218+0000 7fc787fff700 1 -- 172.21.15.6:0/219685784 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc7900036a0 con 0x7fc7a01064c0 2024-03-24T12:04:39.400 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.218+0000 7fc787fff700 1 -- 172.21.15.6:0/219685784 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fc79001a430 con 0x7fc7a01064c0 2024-03-24T12:04:39.400 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.218+0000 7fc7a56c0700 1 -- 172.21.15.6:0/219685784 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a0104470 con 0x7fc7a01064c0 2024-03-24T12:04:39.400 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.218+0000 7fc787fff700 1 -- 172.21.15.6:0/219685784 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc7900193c0 con 0x7fc7a01064c0 2024-03-24T12:04:39.401 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.219+0000 7fc7a56c0700 1 -- 172.21.15.6:0/219685784 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a0104910 con 0x7fc7a01064c0 2024-03-24T12:04:39.401 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.219+0000 7fc787fff700 1 -- 172.21.15.6:0/219685784 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fc790019680 con 0x7fc7a01064c0 2024-03-24T12:04:39.401 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.220+0000 7fc787fff700 1 --2- 172.21.15.6:0/219685784 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7fc788038c60 0x7fc78803b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:39.401 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.220+0000 7fc787fff700 1 -- 172.21.15.6:0/219685784 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc79004d840 con 0x7fc7a01064c0 2024-03-24T12:04:39.401 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.220+0000 7fc79e7fc700 1 --2- 172.21.15.6:0/219685784 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7fc788038c60 0x7fc78803b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:39.401 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.220+0000 7fc7a56c0700 1 -- 172.21.15.6:0/219685784 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a00632a0 con 0x7fc7a01064c0 2024-03-24T12:04:39.401 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.220+0000 7fc79e7fc700 1 --2- 172.21.15.6:0/219685784 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7fc788038c60 0x7fc78803b120 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc794006fd0 tx=0x7fc794006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:39.401 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.225+0000 7fc787fff700 1 -- 172.21.15.6:0/219685784 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc7900199d0 con 0x7fc7a01064c0 2024-03-24T12:04:39.401 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.492+0000 7fc7a56c0700 1 -- 172.21.15.6:0/219685784 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fc7a0109c30 con 0x7fc7a01064c0 2024-03-24T12:04:39.401 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.805+0000 7fc787fff700 1 -- 172.21.15.6:0/219685784 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fc790023c20 con 0x7fc7a01064c0 2024-03-24T12:04:39.401 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.806+0000 7fc787fff700 1 -- 172.21.15.6:0/219685784 <== mon.0 v2:172.21.15.6:3300/0 8 ==== mgrmap(e 9) v1 ==== 45782+0+0 (secure 0 0 0) 0x7fc79002d950 con 0x7fc7a01064c0 2024-03-24T12:04:39.401 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.808+0000 7fc7a56c0700 1 -- 172.21.15.6:0/219685784 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7fc788038c60 msgr2=0x7fc78803b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:39.401 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.808+0000 7fc7a56c0700 1 --2- 172.21.15.6:0/219685784 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7fc788038c60 0x7fc78803b120 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc794006fd0 tx=0x7fc794006e40 comp rx=0 tx=0).stop 2024-03-24T12:04:39.402 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.808+0000 7fc7a56c0700 1 -- 172.21.15.6:0/219685784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7a01064c0 msgr2=0x7fc7a0105d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:39.402 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.808+0000 7fc7a56c0700 1 --2- 172.21.15.6:0/219685784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7a01064c0 0x7fc7a0105d40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc790006af0 tx=0x7fc790011a50 comp rx=0 tx=0).stop 2024-03-24T12:04:39.402 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.808+0000 7fc7a56c0700 1 -- 172.21.15.6:0/219685784 shutdown_connections 2024-03-24T12:04:39.402 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.808+0000 7fc7a56c0700 1 --2- 172.21.15.6:0/219685784 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7fc788038c60 0x7fc78803b120 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:39.402 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.808+0000 7fc7a56c0700 1 --2- 172.21.15.6:0/219685784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc7a01064c0 0x7fc7a0105d40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:39.402 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.808+0000 7fc7a56c0700 1 -- 172.21.15.6:0/219685784 >> 172.21.15.6:0/219685784 conn(0x7fc7a00ffe50 msgr2=0x7fc7a01009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:39.402 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.808+0000 7fc7a56c0700 1 -- 172.21.15.6:0/219685784 shutdown_connections 2024-03-24T12:04:39.402 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:38.808+0000 7fc7a56c0700 1 -- 172.21.15.6:0/219685784 wait complete. 2024-03-24T12:04:39.692 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:39 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:39.382+0000 7f1a07458200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T12:04:39.692 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:39 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:39.485+0000 7f1a07458200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T12:04:39.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/219685784' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-24T12:04:39.948 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:39 smithi006 ceph-mon[30108]: mgrmap e9: y(active, since 20s) 2024-03-24T12:04:40.199 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:40 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:40.156+0000 7f1a07458200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T12:04:40.461 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:40 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:40.254+0000 7f1a07458200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T12:04:40.461 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:40 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:40.400+0000 7f1a07458200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T12:04:40.805 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:40 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:40.507+0000 7f1a07458200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T12:04:40.805 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:40 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:40.593+0000 7f1a07458200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T12:04:41.063 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1541168884' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-24T12:04:41.064 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:40 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:40.862+0000 7f1a07458200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-24T12:04:41.064 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:40 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:40.975+0000 7f1a07458200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T12:04:41.064 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:41 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:41.064+0000 7f1a07458200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T12:04:41.101 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-03-24T12:04:41.101 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-24T12:04:41.101 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-24T12:04:41.101 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-03-24T12:04:41.101 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-24T12:04:41.101 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-03-24T12:04:41.101 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.199+0000 7fe6d238c700 1 Processor -- start 2024-03-24T12:04:41.101 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.199+0000 7fe6d238c700 1 -- start start 2024-03-24T12:04:41.101 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.199+0000 7fe6d238c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6cc104900 0x7fe6cc104d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:41.101 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.199+0000 7fe6d238c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc1052f0 con 0x7fe6cc104900 2024-03-24T12:04:41.102 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.199+0000 7fe6cbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6cc104900 0x7fe6cc104d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:41.102 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.199+0000 7fe6cbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6cc104900 0x7fe6cc104d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51976/0 (socket says 172.21.15.6:51976) 2024-03-24T12:04:41.102 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.199+0000 7fe6cbfff700 1 -- 172.21.15.6:0/975054472 learned_addr learned my addr 172.21.15.6:0/975054472 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:41.102 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.200+0000 7fe6cbfff700 1 -- 172.21.15.6:0/975054472 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6cc105b00 con 0x7fe6cc104900 2024-03-24T12:04:41.102 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.200+0000 7fe6cbfff700 1 --2- 172.21.15.6:0/975054472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6cc104900 0x7fe6cc104d20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc0098a0 tx=0x7fe6bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fdded19267cb357 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:41.102 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.200+0000 7fe6caffd700 1 -- 172.21.15.6:0/975054472 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6bc004160 con 0x7fe6cc104900 2024-03-24T12:04:41.102 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.200+0000 7fe6caffd700 1 -- 172.21.15.6:0/975054472 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe6bc0042c0 con 0x7fe6cc104900 2024-03-24T12:04:41.102 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.200+0000 7fe6caffd700 1 -- 172.21.15.6:0/975054472 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6bc011850 con 0x7fe6cc104900 2024-03-24T12:04:41.102 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.201+0000 7fe6d238c700 1 -- 172.21.15.6:0/975054472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6cc104900 msgr2=0x7fe6cc104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:41.102 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.201+0000 7fe6d238c700 1 --2- 172.21.15.6:0/975054472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6cc104900 0x7fe6cc104d20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc0098a0 tx=0x7fe6bc009bb0 comp rx=0 tx=0).stop 2024-03-24T12:04:41.102 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.201+0000 7fe6d238c700 1 -- 172.21.15.6:0/975054472 shutdown_connections 2024-03-24T12:04:41.102 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.201+0000 7fe6d238c700 1 --2- 172.21.15.6:0/975054472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6cc104900 0x7fe6cc104d20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:41.102 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.201+0000 7fe6d238c700 1 -- 172.21.15.6:0/975054472 >> 172.21.15.6:0/975054472 conn(0x7fe6cc0ffec0 msgr2=0x7fe6cc1022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:41.103 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.201+0000 7fe6d238c700 1 -- 172.21.15.6:0/975054472 shutdown_connections 2024-03-24T12:04:41.103 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.201+0000 7fe6d238c700 1 -- 172.21.15.6:0/975054472 wait complete. 2024-03-24T12:04:41.103 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.202+0000 7fe6d238c700 1 Processor -- start 2024-03-24T12:04:41.103 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.202+0000 7fe6d238c700 1 -- start start 2024-03-24T12:04:41.103 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.202+0000 7fe6d238c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6cc104900 0x7fe6cc18f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:41.103 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.202+0000 7fe6d238c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc18fde0 con 0x7fe6cc104900 2024-03-24T12:04:41.103 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.202+0000 7fe6cbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6cc104900 0x7fe6cc18f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:41.103 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.202+0000 7fe6cbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6cc104900 0x7fe6cc18f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51984/0 (socket says 172.21.15.6:51984) 2024-03-24T12:04:41.103 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.202+0000 7fe6cbfff700 1 -- 172.21.15.6:0/1541168884 learned_addr learned my addr 172.21.15.6:0/1541168884 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:41.103 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.202+0000 7fe6cbfff700 1 -- 172.21.15.6:0/1541168884 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc009550 con 0x7fe6cc104900 2024-03-24T12:04:41.103 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.202+0000 7fe6cbfff700 1 --2- 172.21.15.6:0/1541168884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6cc104900 0x7fe6cc18f8a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc000c00 tx=0x7fe6bc011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:41.103 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.203+0000 7fe6c97fa700 1 -- 172.21.15.6:0/1541168884 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6bc0036a0 con 0x7fe6cc104900 2024-03-24T12:04:41.103 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.203+0000 7fe6c97fa700 1 -- 172.21.15.6:0/1541168884 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe6bc024440 con 0x7fe6cc104900 2024-03-24T12:04:41.103 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.203+0000 7fe6d238c700 1 -- 172.21.15.6:0/1541168884 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6cc18dfd0 con 0x7fe6cc104900 2024-03-24T12:04:41.103 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.204+0000 7fe6d238c700 1 -- 172.21.15.6:0/1541168884 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6cc18e470 con 0x7fe6cc104900 2024-03-24T12:04:41.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.204+0000 7fe6c97fa700 1 -- 172.21.15.6:0/1541168884 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6bc01a430 con 0x7fe6cc104900 2024-03-24T12:04:41.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.204+0000 7fe6c97fa700 1 -- 172.21.15.6:0/1541168884 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 9) v1 ==== 45782+0+0 (secure 0 0 0) 0x7fe6bc003a10 con 0x7fe6cc104900 2024-03-24T12:04:41.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.204+0000 7fe6d238c700 1 -- 172.21.15.6:0/1541168884 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6cc18ee80 con 0x7fe6cc104900 2024-03-24T12:04:41.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.205+0000 7fe6c97fa700 1 --2- 172.21.15.6:0/1541168884 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7fe6b4038960 0x7fe6b403ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:41.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.205+0000 7fe6c97fa700 1 -- 172.21.15.6:0/1541168884 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe6bc04c970 con 0x7fe6cc104900 2024-03-24T12:04:41.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.205+0000 7fe6cb7fe700 1 -- 172.21.15.6:0/1541168884 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7fe6b4038960 msgr2=0x7fe6b403ae20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3010671792 2024-03-24T12:04:41.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.205+0000 7fe6cb7fe700 1 --2- 172.21.15.6:0/1541168884 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7fe6b4038960 0x7fe6b403ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T12:04:41.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.209+0000 7fe6c97fa700 1 -- 172.21.15.6:0/1541168884 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe6bc024820 con 0x7fe6cc104900 2024-03-24T12:04:41.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.406+0000 7fe6cb7fe700 1 -- 172.21.15.6:0/1541168884 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7fe6b4038960 msgr2=0x7fe6b403ae20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3010671792 2024-03-24T12:04:41.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.406+0000 7fe6cb7fe700 1 --2- 172.21.15.6:0/1541168884 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7fe6b4038960 0x7fe6b403ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-24T12:04:41.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.470+0000 7fe6d238c700 1 -- 172.21.15.6:0/1541168884 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fe6cc0632a0 con 0x7fe6cc104900 2024-03-24T12:04:41.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.471+0000 7fe6c97fa700 1 -- 172.21.15.6:0/1541168884 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fe6bc023bf0 con 0x7fe6cc104900 2024-03-24T12:04:41.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.473+0000 7fe6d238c700 1 -- 172.21.15.6:0/1541168884 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7fe6b4038960 msgr2=0x7fe6b403ae20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T12:04:41.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.473+0000 7fe6d238c700 1 --2- 172.21.15.6:0/1541168884 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7fe6b4038960 0x7fe6b403ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:41.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.473+0000 7fe6d238c700 1 -- 172.21.15.6:0/1541168884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6cc104900 msgr2=0x7fe6cc18f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:41.104 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.473+0000 7fe6d238c700 1 --2- 172.21.15.6:0/1541168884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6cc104900 0x7fe6cc18f8a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc000c00 tx=0x7fe6bc011ac0 comp rx=0 tx=0).stop 2024-03-24T12:04:41.105 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.473+0000 7fe6d238c700 1 -- 172.21.15.6:0/1541168884 shutdown_connections 2024-03-24T12:04:41.105 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.473+0000 7fe6d238c700 1 --2- 172.21.15.6:0/1541168884 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7fe6b4038960 0x7fe6b403ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:41.105 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.473+0000 7fe6d238c700 1 --2- 172.21.15.6:0/1541168884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe6cc104900 0x7fe6cc18f8a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:41.105 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.473+0000 7fe6d238c700 1 -- 172.21.15.6:0/1541168884 >> 172.21.15.6:0/1541168884 conn(0x7fe6cc0ffec0 msgr2=0x7fe6cc100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:41.105 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.473+0000 7fe6d238c700 1 -- 172.21.15.6:0/1541168884 shutdown_connections 2024-03-24T12:04:41.105 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:40.473+0000 7fe6d238c700 1 -- 172.21.15.6:0/1541168884 wait complete. 2024-03-24T12:04:41.105 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for the mgr to restart... 2024-03-24T12:04:41.105 INFO:teuthology.orchestra.run.smithi006.stdout:Waiting for mgr epoch 9... 2024-03-24T12:04:41.322 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:41 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:41.165+0000 7f1a07458200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T12:04:41.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:41 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:41.348+0000 7f1a07458200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T12:04:42.461 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:42 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:42.336+0000 7f1a07458200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T12:04:42.806 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:42 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:42.470+0000 7f1a07458200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T12:04:43.098 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:42 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:42.807+0000 7f1a07458200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T12:04:43.099 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:42 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:42.899+0000 7f1a07458200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T12:04:43.099 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:43 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:43.099+0000 7f1a07458200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T12:04:43.461 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:43 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:43.288+0000 7f1a07458200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T12:04:43.840 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:43 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:43.578+0000 7f1a07458200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T12:04:44.211 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:43 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:43.840+0000 7f1a07458200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T12:04:44.211 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:44 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:44.082+0000 7f1a07458200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T12:04:45.211 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:44 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:44.906+0000 7f1a07458200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T12:04:45.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:45.500+0000 7f1a07458200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T12:04:45.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:45.601+0000 7f1a07458200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T12:04:46.211 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:04:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:04:45.822+0000 7f1a07458200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T12:04:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:45 smithi006 ceph-mon[30108]: Active manager daemon y restarted 2024-03-24T12:04:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:45 smithi006 ceph-mon[30108]: Activating manager daemon y 2024-03-24T12:04:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:45 smithi006 ceph-mon[30108]: osdmap e3: 0 total, 0 up, 0 in 2024-03-24T12:04:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:45 smithi006 ceph-mon[30108]: mgrmap e10: y(active, starting, since 0.00304253s) 2024-03-24T12:04:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T12:04:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T12:04:46.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T12:04:46.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T12:04:46.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T12:04:46.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:45 smithi006 ceph-mon[30108]: Manager daemon y is now available 2024-03-24T12:04:46.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:04:46.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:04:47.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:46 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T12:04:47.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:46 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T12:04:47.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:46 smithi006 ceph-mon[30108]: [24/Mar/2024:12:04:46] ENGINE Bus STARTING 2024-03-24T12:04:47.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:46 smithi006 ceph-mon[30108]: [24/Mar/2024:12:04:46] ENGINE Serving on http://172.21.15.6:8765 2024-03-24T12:04:47.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:46 smithi006 ceph-mon[30108]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-24T12:04:47.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:46 smithi006 ceph-mon[30108]: mgrmap e11: y(active, since 1.00537s) 2024-03-24T12:04:47.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:46 smithi006 ceph-mon[30108]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-24T12:04:47.744 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout { 2024-03-24T12:04:47.744 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-24T12:04:47.744 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-24T12:04:47.745 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout } 2024-03-24T12:04:47.745 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.565+0000 7f4600cd4700 1 Processor -- start 2024-03-24T12:04:47.745 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.565+0000 7f4600cd4700 1 -- start start 2024-03-24T12:04:47.745 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.565+0000 7f4600cd4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45fc104930 0x7f45fc104d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:47.745 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.565+0000 7f4600cd4700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc105320 con 0x7f45fc104930 2024-03-24T12:04:47.745 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.565+0000 7f45fa59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45fc104930 0x7f45fc104d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:47.745 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.565+0000 7f45fa59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45fc104930 0x7f45fc104d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51990/0 (socket says 172.21.15.6:51990) 2024-03-24T12:04:47.745 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.565+0000 7f45fa59c700 1 -- 172.21.15.6:0/2603538499 learned_addr learned my addr 172.21.15.6:0/2603538499 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:47.745 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.566+0000 7f45fa59c700 1 -- 172.21.15.6:0/2603538499 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45fc105b30 con 0x7f45fc104930 2024-03-24T12:04:47.745 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.566+0000 7f45fa59c700 1 --2- 172.21.15.6:0/2603538499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45fc104930 0x7f45fc104d50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f45ec0098a0 tx=0x7f45ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb3afae9cdacb8a server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:47.745 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.566+0000 7f45f959a700 1 -- 172.21.15.6:0/2603538499 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45ec004160 con 0x7f45fc104930 2024-03-24T12:04:47.745 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.566+0000 7f45f959a700 1 -- 172.21.15.6:0/2603538499 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f45ec0042c0 con 0x7f45fc104930 2024-03-24T12:04:47.745 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.566+0000 7f45f959a700 1 -- 172.21.15.6:0/2603538499 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45ec011850 con 0x7f45fc104930 2024-03-24T12:04:47.746 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.567+0000 7f4600cd4700 1 -- 172.21.15.6:0/2603538499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45fc104930 msgr2=0x7f45fc104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:47.746 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.567+0000 7f4600cd4700 1 --2- 172.21.15.6:0/2603538499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45fc104930 0x7f45fc104d50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f45ec0098a0 tx=0x7f45ec009bb0 comp rx=0 tx=0).stop 2024-03-24T12:04:47.746 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.567+0000 7f4600cd4700 1 -- 172.21.15.6:0/2603538499 shutdown_connections 2024-03-24T12:04:47.746 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.567+0000 7f4600cd4700 1 --2- 172.21.15.6:0/2603538499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45fc104930 0x7f45fc104d50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:47.746 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.567+0000 7f4600cd4700 1 -- 172.21.15.6:0/2603538499 >> 172.21.15.6:0/2603538499 conn(0x7f45fc0ffef0 msgr2=0x7f45fc102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:47.746 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.567+0000 7f4600cd4700 1 -- 172.21.15.6:0/2603538499 shutdown_connections 2024-03-24T12:04:47.746 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.567+0000 7f4600cd4700 1 -- 172.21.15.6:0/2603538499 wait complete. 2024-03-24T12:04:47.746 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.568+0000 7f4600cd4700 1 Processor -- start 2024-03-24T12:04:47.746 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.568+0000 7f4600cd4700 1 -- start start 2024-03-24T12:04:47.746 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.568+0000 7f4600cd4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45fc104930 0x7f45fc199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:47.746 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.568+0000 7f4600cd4700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc199c40 con 0x7f45fc104930 2024-03-24T12:04:47.746 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.568+0000 7f45fa59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45fc104930 0x7f45fc199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:47.746 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.568+0000 7f45fa59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45fc104930 0x7f45fc199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:52002/0 (socket says 172.21.15.6:52002) 2024-03-24T12:04:47.747 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.568+0000 7f45fa59c700 1 -- 172.21.15.6:0/1874447315 learned_addr learned my addr 172.21.15.6:0/1874447315 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:47.747 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.568+0000 7f45fa59c700 1 -- 172.21.15.6:0/1874447315 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ec009550 con 0x7f45fc104930 2024-03-24T12:04:47.747 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.568+0000 7f45fa59c700 1 --2- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45fc104930 0x7f45fc199700 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f45ec003d50 tx=0x7f45ec011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:47.747 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.569+0000 7f45eb7fe700 1 -- 172.21.15.6:0/1874447315 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45ec0037d0 con 0x7f45fc104930 2024-03-24T12:04:47.747 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.569+0000 7f4600cd4700 1 -- 172.21.15.6:0/1874447315 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45fc199e40 con 0x7f45fc104930 2024-03-24T12:04:47.747 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.569+0000 7f45eb7fe700 1 -- 172.21.15.6:0/1874447315 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f45ec003930 con 0x7f45fc104930 2024-03-24T12:04:47.747 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.569+0000 7f45eb7fe700 1 -- 172.21.15.6:0/1874447315 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45ec0193c0 con 0x7f45fc104930 2024-03-24T12:04:47.747 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.569+0000 7f4600cd4700 1 -- 172.21.15.6:0/1874447315 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45fc19a2e0 con 0x7f45fc104930 2024-03-24T12:04:47.747 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.569+0000 7f45eb7fe700 1 -- 172.21.15.6:0/1874447315 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 9) v1 ==== 45782+0+0 (secure 0 0 0) 0x7f45ec019680 con 0x7f45fc104930 2024-03-24T12:04:47.747 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.570+0000 7f45eb7fe700 1 --2- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f45e4038d00 0x7f45e403b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:47.747 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.570+0000 7f4600cd4700 1 -- 172.21.15.6:0/1874447315 --> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f45fc0632a0 con 0x7f45e4038d00 2024-03-24T12:04:47.747 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.570+0000 7f45eb7fe700 1 -- 172.21.15.6:0/1874447315 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f45ec04ccd0 con 0x7f45fc104930 2024-03-24T12:04:47.747 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.570+0000 7f45f9d9b700 1 -- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f45e4038d00 msgr2=0x7f45e403b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3010671792 2024-03-24T12:04:47.747 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.570+0000 7f45f9d9b700 1 --2- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f45e4038d00 0x7f45e403b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T12:04:47.748 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.771+0000 7f45f9d9b700 1 -- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f45e4038d00 msgr2=0x7f45e403b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3010671792 2024-03-24T12:04:47.748 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:41.771+0000 7f45f9d9b700 1 --2- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f45e4038d00 0x7f45e403b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-24T12:04:47.748 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:42.172+0000 7f45f9d9b700 1 -- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f45e4038d00 msgr2=0x7f45e403b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3010671792 2024-03-24T12:04:47.748 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:42.172+0000 7f45f9d9b700 1 --2- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f45e4038d00 0x7f45e403b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-24T12:04:47.748 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:42.973+0000 7f45f9d9b700 1 -- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f45e4038d00 msgr2=0x7f45e403b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3010671792 2024-03-24T12:04:47.748 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:42.973+0000 7f45f9d9b700 1 --2- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f45e4038d00 0x7f45e403b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-24T12:04:47.748 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:44.574+0000 7f45f9d9b700 1 -- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f45e4038d00 msgr2=0x7f45e403b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3010671792 2024-03-24T12:04:47.748 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:44.574+0000 7f45f9d9b700 1 --2- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f45e4038d00 0x7f45e403b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-24T12:04:47.748 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:45.838+0000 7f45eb7fe700 1 -- 172.21.15.6:0/1874447315 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mgrmap(e 10) v1 ==== 45549+0+0 (secure 0 0 0) 0x7f45ec02c8c0 con 0x7f45fc104930 2024-03-24T12:04:47.748 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:45.838+0000 7f45eb7fe700 1 -- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f45e4038d00 msgr2=0x7f45e403b1c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T12:04:47.748 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:45.838+0000 7f45eb7fe700 1 --2- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f45e4038d00 0x7f45e403b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:47.748 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:46.839+0000 7f45eb7fe700 1 -- 172.21.15.6:0/1874447315 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 11) v1 ==== 45676+0+0 (secure 0 0 0) 0x7f45ec04e4a0 con 0x7f45fc104930 2024-03-24T12:04:47.748 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:46.839+0000 7f45eb7fe700 1 --2- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f45e403c620 0x7f45e403ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:47.749 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:46.840+0000 7f45eb7fe700 1 -- 172.21.15.6:0/1874447315 --> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f45fc0632a0 con 0x7f45e403c620 2024-03-24T12:04:47.749 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:46.840+0000 7f45f9d9b700 1 --2- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f45e403c620 0x7f45e403ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:47.749 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:46.841+0000 7f45f9d9b700 1 --2- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f45e403c620 0x7f45e403ea10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f45f0003a10 tx=0x7f45f00092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:47.749 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:46.841+0000 7f45eb7fe700 1 -- 172.21.15.6:0/1874447315 <== mgr.14152 v2:172.21.15.6:6800/3847865179 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f45fc0632a0 con 0x7f45e403c620 2024-03-24T12:04:47.749 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:46.848+0000 7f4600cd4700 1 -- 172.21.15.6:0/1874447315 --> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f45fc108d60 con 0x7f45e403c620 2024-03-24T12:04:47.749 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:46.848+0000 7f45eb7fe700 1 -- 172.21.15.6:0/1874447315 <== mgr.14152 v2:172.21.15.6:6800/3847865179 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f45fc108d60 con 0x7f45e403c620 2024-03-24T12:04:47.749 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:46.848+0000 7f4600cd4700 1 -- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f45e403c620 msgr2=0x7f45e403ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:47.749 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:46.848+0000 7f4600cd4700 1 --2- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f45e403c620 0x7f45e403ea10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f45f0003a10 tx=0x7f45f00092b0 comp rx=0 tx=0).stop 2024-03-24T12:04:47.749 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:46.848+0000 7f4600cd4700 1 -- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45fc104930 msgr2=0x7f45fc199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:47.749 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:46.848+0000 7f4600cd4700 1 --2- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45fc104930 0x7f45fc199700 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f45ec003d50 tx=0x7f45ec011ac0 comp rx=0 tx=0).stop 2024-03-24T12:04:47.749 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:46.848+0000 7f4600cd4700 1 -- 172.21.15.6:0/1874447315 shutdown_connections 2024-03-24T12:04:47.749 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:46.848+0000 7f4600cd4700 1 --2- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f45e403c620 0x7f45e403ea10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:47.749 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:46.848+0000 7f4600cd4700 1 --2- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45fc104930 0x7f45fc199700 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:47.749 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:46.848+0000 7f4600cd4700 1 --2- 172.21.15.6:0/1874447315 >> [v2:172.21.15.6:6800/3010671792,v1:172.21.15.6:6801/3010671792] conn(0x7f45e4038d00 0x7f45e403b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:47.750 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:46.848+0000 7f4600cd4700 1 -- 172.21.15.6:0/1874447315 >> 172.21.15.6:0/1874447315 conn(0x7f45fc0ffef0 msgr2=0x7f45fc100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:47.750 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:46.848+0000 7f4600cd4700 1 -- 172.21.15.6:0/1874447315 shutdown_connections 2024-03-24T12:04:47.750 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:46.848+0000 7f4600cd4700 1 -- 172.21.15.6:0/1874447315 wait complete. 2024-03-24T12:04:47.750 INFO:teuthology.orchestra.run.smithi006.stdout:mgr epoch 9 is available 2024-03-24T12:04:47.750 INFO:teuthology.orchestra.run.smithi006.stdout:Generating a dashboard self-signed certificate... 2024-03-24T12:04:48.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:48 smithi006 ceph-mon[30108]: [24/Mar/2024:12:04:46] ENGINE Serving on https://172.21.15.6:7150 2024-03-24T12:04:48.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:48 smithi006 ceph-mon[30108]: [24/Mar/2024:12:04:46] ENGINE Bus STARTED 2024-03-24T12:04:48.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:48 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:04:49.032 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-24T12:04:49.033 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.217+0000 7fca40e14700 1 Processor -- start 2024-03-24T12:04:49.033 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.217+0000 7fca40e14700 1 -- start start 2024-03-24T12:04:49.033 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.218+0000 7fca40e14700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca3c105eb0 0x7fca3c1062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:49.033 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.218+0000 7fca40e14700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c1068a0 con 0x7fca3c105eb0 2024-03-24T12:04:49.033 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.218+0000 7fca3a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca3c105eb0 0x7fca3c1062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:49.033 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.218+0000 7fca3a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca3c105eb0 0x7fca3c1062d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54836/0 (socket says 172.21.15.6:54836) 2024-03-24T12:04:49.033 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.218+0000 7fca3a59c700 1 -- 172.21.15.6:0/3769061762 learned_addr learned my addr 172.21.15.6:0/3769061762 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:49.033 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.218+0000 7fca3a59c700 1 -- 172.21.15.6:0/3769061762 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c104b10 con 0x7fca3c105eb0 2024-03-24T12:04:49.033 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.218+0000 7fca3a59c700 1 --2- 172.21.15.6:0/3769061762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca3c105eb0 0x7fca3c1062d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0098a0 tx=0x7fca2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e662e44eb36de8a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:49.033 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.219+0000 7fca3959a700 1 -- 172.21.15.6:0/3769061762 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca2c004160 con 0x7fca3c105eb0 2024-03-24T12:04:49.033 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.219+0000 7fca3959a700 1 -- 172.21.15.6:0/3769061762 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fca2c0042c0 con 0x7fca3c105eb0 2024-03-24T12:04:49.034 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.219+0000 7fca3959a700 1 -- 172.21.15.6:0/3769061762 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca2c011850 con 0x7fca3c105eb0 2024-03-24T12:04:49.034 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.219+0000 7fca40e14700 1 -- 172.21.15.6:0/3769061762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca3c105eb0 msgr2=0x7fca3c1062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:49.034 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.219+0000 7fca40e14700 1 --2- 172.21.15.6:0/3769061762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca3c105eb0 0x7fca3c1062d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0098a0 tx=0x7fca2c009bb0 comp rx=0 tx=0).stop 2024-03-24T12:04:49.034 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.219+0000 7fca40e14700 1 -- 172.21.15.6:0/3769061762 shutdown_connections 2024-03-24T12:04:49.034 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.219+0000 7fca40e14700 1 --2- 172.21.15.6:0/3769061762 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca3c105eb0 0x7fca3c1062d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:49.034 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.219+0000 7fca40e14700 1 -- 172.21.15.6:0/3769061762 >> 172.21.15.6:0/3769061762 conn(0x7fca3c0ff640 msgr2=0x7fca3c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:49.034 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.219+0000 7fca40e14700 1 -- 172.21.15.6:0/3769061762 shutdown_connections 2024-03-24T12:04:49.034 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.219+0000 7fca40e14700 1 -- 172.21.15.6:0/3769061762 wait complete. 2024-03-24T12:04:49.034 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.220+0000 7fca40e14700 1 Processor -- start 2024-03-24T12:04:49.034 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.220+0000 7fca40e14700 1 -- start start 2024-03-24T12:04:49.034 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.220+0000 7fca40e14700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca3c105eb0 0x7fca3c1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:49.034 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.220+0000 7fca40e14700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c199c30 con 0x7fca3c105eb0 2024-03-24T12:04:49.034 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.220+0000 7fca3a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca3c105eb0 0x7fca3c1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:49.034 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.220+0000 7fca3a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca3c105eb0 0x7fca3c1996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54846/0 (socket says 172.21.15.6:54846) 2024-03-24T12:04:49.035 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.220+0000 7fca3a59c700 1 -- 172.21.15.6:0/1535257128 learned_addr learned my addr 172.21.15.6:0/1535257128 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:49.035 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.221+0000 7fca3a59c700 1 -- 172.21.15.6:0/1535257128 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2c009550 con 0x7fca3c105eb0 2024-03-24T12:04:49.035 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.221+0000 7fca3a59c700 1 --2- 172.21.15.6:0/1535257128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca3c105eb0 0x7fca3c1996f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fca2c006af0 tx=0x7fca2c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:49.035 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.221+0000 7fca337fe700 1 -- 172.21.15.6:0/1535257128 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca2c0036a0 con 0x7fca3c105eb0 2024-03-24T12:04:49.035 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.221+0000 7fca337fe700 1 -- 172.21.15.6:0/1535257128 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fca2c01a430 con 0x7fca3c105eb0 2024-03-24T12:04:49.035 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.221+0000 7fca40e14700 1 -- 172.21.15.6:0/1535257128 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3c199e30 con 0x7fca3c105eb0 2024-03-24T12:04:49.035 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.221+0000 7fca337fe700 1 -- 172.21.15.6:0/1535257128 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca2c01ae60 con 0x7fca3c105eb0 2024-03-24T12:04:49.035 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.221+0000 7fca40e14700 1 -- 172.21.15.6:0/1535257128 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3c19a2d0 con 0x7fca3c105eb0 2024-03-24T12:04:49.035 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.222+0000 7fca337fe700 1 -- 172.21.15.6:0/1535257128 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 11) v1 ==== 45676+0+0 (secure 0 0 0) 0x7fca2c0194a0 con 0x7fca3c105eb0 2024-03-24T12:04:49.035 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.222+0000 7fca337fe700 1 --2- 172.21.15.6:0/1535257128 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fca28038ba0 0x7fca2803b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:49.035 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.222+0000 7fca337fe700 1 -- 172.21.15.6:0/1535257128 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fca2c003800 con 0x7fca3c105eb0 2024-03-24T12:04:49.035 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.222+0000 7fca39d9b700 1 --2- 172.21.15.6:0/1535257128 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fca28038ba0 0x7fca2803b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:49.035 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.222+0000 7fca40e14700 1 -- 172.21.15.6:0/1535257128 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3c1937f0 con 0x7fca3c105eb0 2024-03-24T12:04:49.035 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.222+0000 7fca39d9b700 1 --2- 172.21.15.6:0/1535257128 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fca28038ba0 0x7fca2803b060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fca24006fd0 tx=0x7fca24006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:49.035 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.227+0000 7fca337fe700 1 -- 172.21.15.6:0/1535257128 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fca2c02d440 con 0x7fca3c105eb0 2024-03-24T12:04:49.036 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.437+0000 7fca40e14700 1 -- 172.21.15.6:0/1535257128 --> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fca3c0620b0 con 0x7fca28038ba0 2024-03-24T12:04:49.036 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.456+0000 7fca337fe700 1 -- 172.21.15.6:0/1535257128 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 12) v1 ==== 45782+0+0 (secure 0 0 0) 0x7fca2c003a10 con 0x7fca3c105eb0 2024-03-24T12:04:49.036 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.489+0000 7fca337fe700 1 -- 172.21.15.6:0/1535257128 <== mgr.14152 v2:172.21.15.6:6800/3847865179 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fca3c0620b0 con 0x7fca28038ba0 2024-03-24T12:04:49.036 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.491+0000 7fca40e14700 1 -- 172.21.15.6:0/1535257128 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fca28038ba0 msgr2=0x7fca2803b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:49.036 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.491+0000 7fca40e14700 1 --2- 172.21.15.6:0/1535257128 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fca28038ba0 0x7fca2803b060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fca24006fd0 tx=0x7fca24006e40 comp rx=0 tx=0).stop 2024-03-24T12:04:49.036 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.491+0000 7fca40e14700 1 -- 172.21.15.6:0/1535257128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca3c105eb0 msgr2=0x7fca3c1996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:49.036 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.491+0000 7fca40e14700 1 --2- 172.21.15.6:0/1535257128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca3c105eb0 0x7fca3c1996f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fca2c006af0 tx=0x7fca2c011a30 comp rx=0 tx=0).stop 2024-03-24T12:04:49.036 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.491+0000 7fca40e14700 1 -- 172.21.15.6:0/1535257128 shutdown_connections 2024-03-24T12:04:49.036 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.491+0000 7fca40e14700 1 --2- 172.21.15.6:0/1535257128 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fca28038ba0 0x7fca2803b060 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:49.036 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.491+0000 7fca40e14700 1 --2- 172.21.15.6:0/1535257128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca3c105eb0 0x7fca3c1996f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:49.036 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.491+0000 7fca40e14700 1 -- 172.21.15.6:0/1535257128 >> 172.21.15.6:0/1535257128 conn(0x7fca3c0ff640 msgr2=0x7fca3c100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:49.036 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.491+0000 7fca40e14700 1 -- 172.21.15.6:0/1535257128 shutdown_connections 2024-03-24T12:04:49.036 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:48.491+0000 7fca40e14700 1 -- 172.21.15.6:0/1535257128 wait complete. 2024-03-24T12:04:49.036 INFO:teuthology.orchestra.run.smithi006.stdout:Creating initial admin user... 2024-03-24T12:04:49.676 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:49 smithi006 ceph-mon[30108]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:04:49.676 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:49 smithi006 ceph-mon[30108]: mgrmap e12: y(active, since 2s) 2024-03-24T12:04:49.676 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:49 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:04:49.677 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:49 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:04:49.677 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:49 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:04:50.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:50 smithi006 ceph-mon[30108]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:04:50.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:50 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:04:51.239 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$e9O.cTOkbWKZLRWEHOYiuux9W1mnEMN3NTQ/y1qW17tSmuuc7k7V6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711281889, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-24T12:04:51.241 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.445+0000 7f4923924700 1 Processor -- start 2024-03-24T12:04:51.241 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.445+0000 7f4923924700 1 -- start start 2024-03-24T12:04:51.241 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.446+0000 7f4923924700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f491c103cf0 0x7f491c104110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:51.241 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.446+0000 7f4923924700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c1046e0 con 0x7f491c103cf0 2024-03-24T12:04:51.241 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.446+0000 7f49216c0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f491c103cf0 0x7f491c104110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:51.241 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.446+0000 7f49216c0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f491c103cf0 0x7f491c104110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54854/0 (socket says 172.21.15.6:54854) 2024-03-24T12:04:51.241 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.446+0000 7f49216c0700 1 -- 172.21.15.6:0/4027705927 learned_addr learned my addr 172.21.15.6:0/4027705927 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:51.242 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.446+0000 7f49216c0700 1 -- 172.21.15.6:0/4027705927 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491c106c80 con 0x7f491c103cf0 2024-03-24T12:04:51.242 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.446+0000 7f49216c0700 1 --2- 172.21.15.6:0/4027705927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f491c103cf0 0x7f491c104110 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f49180098a0 tx=0x7f4918009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d730d9ca33b0f4f server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:51.242 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.446+0000 7f4913fff700 1 -- 172.21.15.6:0/4027705927 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4918004160 con 0x7f491c103cf0 2024-03-24T12:04:51.242 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.446+0000 7f4913fff700 1 -- 172.21.15.6:0/4027705927 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f49180042c0 con 0x7f491c103cf0 2024-03-24T12:04:51.242 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.446+0000 7f4913fff700 1 -- 172.21.15.6:0/4027705927 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4918011850 con 0x7f491c103cf0 2024-03-24T12:04:51.242 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.447+0000 7f4923924700 1 -- 172.21.15.6:0/4027705927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f491c103cf0 msgr2=0x7f491c104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:51.242 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.447+0000 7f4923924700 1 --2- 172.21.15.6:0/4027705927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f491c103cf0 0x7f491c104110 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f49180098a0 tx=0x7f4918009bb0 comp rx=0 tx=0).stop 2024-03-24T12:04:51.242 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.447+0000 7f4923924700 1 -- 172.21.15.6:0/4027705927 shutdown_connections 2024-03-24T12:04:51.242 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.447+0000 7f4923924700 1 --2- 172.21.15.6:0/4027705927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f491c103cf0 0x7f491c104110 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:51.242 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.447+0000 7f4923924700 1 -- 172.21.15.6:0/4027705927 >> 172.21.15.6:0/4027705927 conn(0x7f491c0ff640 msgr2=0x7f491c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:51.242 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.447+0000 7f4923924700 1 -- 172.21.15.6:0/4027705927 shutdown_connections 2024-03-24T12:04:51.242 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.447+0000 7f4923924700 1 -- 172.21.15.6:0/4027705927 wait complete. 2024-03-24T12:04:51.242 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.448+0000 7f4923924700 1 Processor -- start 2024-03-24T12:04:51.242 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.448+0000 7f4923924700 1 -- start start 2024-03-24T12:04:51.243 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.448+0000 7f4923924700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f491c103cf0 0x7f491c199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:51.243 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.448+0000 7f4923924700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c199bd0 con 0x7f491c103cf0 2024-03-24T12:04:51.243 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.448+0000 7f49216c0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f491c103cf0 0x7f491c199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:51.243 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.448+0000 7f49216c0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f491c103cf0 0x7f491c199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54862/0 (socket says 172.21.15.6:54862) 2024-03-24T12:04:51.243 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.448+0000 7f49216c0700 1 -- 172.21.15.6:0/882323763 learned_addr learned my addr 172.21.15.6:0/882323763 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:51.243 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.448+0000 7f49216c0700 1 -- 172.21.15.6:0/882323763 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4918009550 con 0x7f491c103cf0 2024-03-24T12:04:51.243 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.449+0000 7f49216c0700 1 --2- 172.21.15.6:0/882323763 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f491c103cf0 0x7f491c199690 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4918000c00 tx=0x7f4918011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:51.243 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.449+0000 7f49127fc700 1 -- 172.21.15.6:0/882323763 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49180036a0 con 0x7f491c103cf0 2024-03-24T12:04:51.243 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.449+0000 7f49127fc700 1 -- 172.21.15.6:0/882323763 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f4918024440 con 0x7f491c103cf0 2024-03-24T12:04:51.243 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.449+0000 7f49127fc700 1 -- 172.21.15.6:0/882323763 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f491801a430 con 0x7f491c103cf0 2024-03-24T12:04:51.243 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.449+0000 7f4923924700 1 -- 172.21.15.6:0/882323763 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491c199dd0 con 0x7f491c103cf0 2024-03-24T12:04:51.243 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.449+0000 7f4923924700 1 -- 172.21.15.6:0/882323763 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f491c19a1b0 con 0x7f491c103cf0 2024-03-24T12:04:51.243 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.450+0000 7f49127fc700 1 -- 172.21.15.6:0/882323763 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 12) v1 ==== 45782+0+0 (secure 0 0 0) 0x7f4918024820 con 0x7f491c103cf0 2024-03-24T12:04:51.243 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.450+0000 7f4923924700 1 -- 172.21.15.6:0/882323763 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f491c0632a0 con 0x7f491c103cf0 2024-03-24T12:04:51.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.450+0000 7f49127fc700 1 --2- 172.21.15.6:0/882323763 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f4908038d60 0x7f490803b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:51.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.450+0000 7f49127fc700 1 -- 172.21.15.6:0/882323763 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f491804cec0 con 0x7f491c103cf0 2024-03-24T12:04:51.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.451+0000 7f4920ebf700 1 --2- 172.21.15.6:0/882323763 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f4908038d60 0x7f490803b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:51.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.451+0000 7f4920ebf700 1 --2- 172.21.15.6:0/882323763 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f4908038d60 0x7f490803b220 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f490c006fd0 tx=0x7f490c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:51.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.456+0000 7f49127fc700 1 -- 172.21.15.6:0/882323763 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4918018be0 con 0x7f491c103cf0 2024-03-24T12:04:51.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.691+0000 7f4923924700 1 -- 172.21.15.6:0/882323763 --> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f491c109eb0 con 0x7f4908038d60 2024-03-24T12:04:51.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.937+0000 7f49127fc700 1 -- 172.21.15.6:0/882323763 <== mgr.14152 v2:172.21.15.6:6800/3847865179 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f491c109eb0 con 0x7f4908038d60 2024-03-24T12:04:51.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.939+0000 7f4923924700 1 -- 172.21.15.6:0/882323763 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f4908038d60 msgr2=0x7f490803b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:51.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.939+0000 7f4923924700 1 --2- 172.21.15.6:0/882323763 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f4908038d60 0x7f490803b220 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f490c006fd0 tx=0x7f490c006e40 comp rx=0 tx=0).stop 2024-03-24T12:04:51.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.939+0000 7f4923924700 1 -- 172.21.15.6:0/882323763 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f491c103cf0 msgr2=0x7f491c199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:51.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.939+0000 7f4923924700 1 --2- 172.21.15.6:0/882323763 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f491c103cf0 0x7f491c199690 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4918000c00 tx=0x7f4918011ac0 comp rx=0 tx=0).stop 2024-03-24T12:04:51.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.939+0000 7f4923924700 1 -- 172.21.15.6:0/882323763 shutdown_connections 2024-03-24T12:04:51.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.939+0000 7f4923924700 1 --2- 172.21.15.6:0/882323763 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f4908038d60 0x7f490803b220 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:51.244 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.939+0000 7f4923924700 1 --2- 172.21.15.6:0/882323763 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f491c103cf0 0x7f491c199690 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:51.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.939+0000 7f4923924700 1 -- 172.21.15.6:0/882323763 >> 172.21.15.6:0/882323763 conn(0x7f491c0ff640 msgr2=0x7f491c1001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:51.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.939+0000 7f4923924700 1 -- 172.21.15.6:0/882323763 shutdown_connections 2024-03-24T12:04:51.245 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:49.939+0000 7f4923924700 1 -- 172.21.15.6:0/882323763 wait complete. 2024-03-24T12:04:51.245 INFO:teuthology.orchestra.run.smithi006.stdout:Fetching dashboard port number... 2024-03-24T12:04:52.185 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/17635734' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-24T12:04:52.185 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:51 smithi006 ceph-mon[30108]: mgrmap e13: y(active, since 6s) 2024-03-24T12:04:52.435 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stdout 8443 2024-03-24T12:04:52.435 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.678+0000 7f6b1a8dc700 1 Processor -- start 2024-03-24T12:04:52.435 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.678+0000 7f6b1a8dc700 1 -- start start 2024-03-24T12:04:52.435 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.678+0000 7f6b1a8dc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b141070d0 0x7f6b141074f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:52.435 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.678+0000 7f6b1a8dc700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b14107ac0 con 0x7f6b141070d0 2024-03-24T12:04:52.435 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.679+0000 7f6b13fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b141070d0 0x7f6b141074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:52.435 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.679+0000 7f6b13fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b141070d0 0x7f6b141074f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54876/0 (socket says 172.21.15.6:54876) 2024-03-24T12:04:52.435 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.679+0000 7f6b13fff700 1 -- 172.21.15.6:0/3185365159 learned_addr learned my addr 172.21.15.6:0/3185365159 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:52.435 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.679+0000 7f6b13fff700 1 -- 172.21.15.6:0/3185365159 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b14105610 con 0x7f6b141070d0 2024-03-24T12:04:52.435 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.679+0000 7f6b13fff700 1 --2- 172.21.15.6:0/3185365159 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b141070d0 0x7f6b141074f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6b040098a0 tx=0x7f6b04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269b5387a4a52691 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:52.436 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.679+0000 7f6b12ffd700 1 -- 172.21.15.6:0/3185365159 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b04004160 con 0x7f6b141070d0 2024-03-24T12:04:52.436 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.679+0000 7f6b12ffd700 1 -- 172.21.15.6:0/3185365159 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f6b040042c0 con 0x7f6b141070d0 2024-03-24T12:04:52.436 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.679+0000 7f6b12ffd700 1 -- 172.21.15.6:0/3185365159 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b04011850 con 0x7f6b141070d0 2024-03-24T12:04:52.436 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.680+0000 7f6b1a8dc700 1 -- 172.21.15.6:0/3185365159 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b141070d0 msgr2=0x7f6b141074f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:52.436 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.680+0000 7f6b1a8dc700 1 --2- 172.21.15.6:0/3185365159 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b141070d0 0x7f6b141074f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6b040098a0 tx=0x7f6b04009bb0 comp rx=0 tx=0).stop 2024-03-24T12:04:52.436 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.680+0000 7f6b1a8dc700 1 -- 172.21.15.6:0/3185365159 shutdown_connections 2024-03-24T12:04:52.436 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.680+0000 7f6b1a8dc700 1 --2- 172.21.15.6:0/3185365159 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b141070d0 0x7f6b141074f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:52.436 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.680+0000 7f6b1a8dc700 1 -- 172.21.15.6:0/3185365159 >> 172.21.15.6:0/3185365159 conn(0x7f6b14100f50 msgr2=0x7f6b141033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:52.436 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.680+0000 7f6b1a8dc700 1 -- 172.21.15.6:0/3185365159 shutdown_connections 2024-03-24T12:04:52.436 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.680+0000 7f6b1a8dc700 1 -- 172.21.15.6:0/3185365159 wait complete. 2024-03-24T12:04:52.436 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.680+0000 7f6b1a8dc700 1 Processor -- start 2024-03-24T12:04:52.436 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.680+0000 7f6b1a8dc700 1 -- start start 2024-03-24T12:04:52.436 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.681+0000 7f6b1a8dc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b141070d0 0x7f6b14199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:52.436 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.681+0000 7f6b1a8dc700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b14199cd0 con 0x7f6b141070d0 2024-03-24T12:04:52.437 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.681+0000 7f6b13fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b141070d0 0x7f6b14199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:52.437 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.681+0000 7f6b13fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b141070d0 0x7f6b14199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54882/0 (socket says 172.21.15.6:54882) 2024-03-24T12:04:52.437 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.681+0000 7f6b13fff700 1 -- 172.21.15.6:0/17635734 learned_addr learned my addr 172.21.15.6:0/17635734 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:52.437 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.681+0000 7f6b13fff700 1 -- 172.21.15.6:0/17635734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b04009550 con 0x7f6b141070d0 2024-03-24T12:04:52.437 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.681+0000 7f6b13fff700 1 --2- 172.21.15.6:0/17635734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b141070d0 0x7f6b14199790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6b04003d50 tx=0x7f6b04011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:52.437 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.681+0000 7f6b117fa700 1 -- 172.21.15.6:0/17635734 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b040037d0 con 0x7f6b141070d0 2024-03-24T12:04:52.437 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.681+0000 7f6b117fa700 1 -- 172.21.15.6:0/17635734 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f6b04003930 con 0x7f6b141070d0 2024-03-24T12:04:52.437 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.681+0000 7f6b1a8dc700 1 -- 172.21.15.6:0/17635734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b14199ed0 con 0x7f6b141070d0 2024-03-24T12:04:52.437 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.682+0000 7f6b117fa700 1 -- 172.21.15.6:0/17635734 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b040193c0 con 0x7f6b141070d0 2024-03-24T12:04:52.437 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.682+0000 7f6b1a8dc700 1 -- 172.21.15.6:0/17635734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b1419a370 con 0x7f6b141070d0 2024-03-24T12:04:52.437 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.682+0000 7f6b117fa700 1 -- 172.21.15.6:0/17635734 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 12) v1 ==== 45782+0+0 (secure 0 0 0) 0x7f6b04019630 con 0x7f6b141070d0 2024-03-24T12:04:52.437 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.682+0000 7f6b1a8dc700 1 -- 172.21.15.6:0/17635734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b140632a0 con 0x7f6b141070d0 2024-03-24T12:04:52.437 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.683+0000 7f6b117fa700 1 --2- 172.21.15.6:0/17635734 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f6afc038d50 0x7f6afc03b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:52.438 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.683+0000 7f6b117fa700 1 -- 172.21.15.6:0/17635734 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6b0404ce30 con 0x7f6b141070d0 2024-03-24T12:04:52.438 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.683+0000 7f6b137fe700 1 --2- 172.21.15.6:0/17635734 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f6afc038d50 0x7f6afc03b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:52.438 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.683+0000 7f6b137fe700 1 --2- 172.21.15.6:0/17635734 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f6afc038d50 0x7f6afc03b210 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6b08006fd0 tx=0x7f6b08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:52.438 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.687+0000 7f6b117fa700 1 -- 172.21.15.6:0/17635734 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6b04010c50 con 0x7f6b141070d0 2024-03-24T12:04:52.438 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.888+0000 7f6b1a8dc700 1 -- 172.21.15.6:0/17635734 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f6b1419cc50 con 0x7f6b141070d0 2024-03-24T12:04:52.438 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.888+0000 7f6b117fa700 1 -- 172.21.15.6:0/17635734 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f6b0402be20 con 0x7f6b141070d0 2024-03-24T12:04:52.438 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.890+0000 7f6b1a8dc700 1 -- 172.21.15.6:0/17635734 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f6afc038d50 msgr2=0x7f6afc03b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:52.438 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.890+0000 7f6b1a8dc700 1 --2- 172.21.15.6:0/17635734 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f6afc038d50 0x7f6afc03b210 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6b08006fd0 tx=0x7f6b08006e40 comp rx=0 tx=0).stop 2024-03-24T12:04:52.438 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.890+0000 7f6b1a8dc700 1 -- 172.21.15.6:0/17635734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b141070d0 msgr2=0x7f6b14199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:52.438 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.890+0000 7f6b1a8dc700 1 --2- 172.21.15.6:0/17635734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b141070d0 0x7f6b14199790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6b04003d50 tx=0x7f6b04011ac0 comp rx=0 tx=0).stop 2024-03-24T12:04:52.438 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.891+0000 7f6b1a8dc700 1 -- 172.21.15.6:0/17635734 shutdown_connections 2024-03-24T12:04:52.438 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.891+0000 7f6b1a8dc700 1 --2- 172.21.15.6:0/17635734 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f6afc038d50 0x7f6afc03b210 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:52.438 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.891+0000 7f6b1a8dc700 1 --2- 172.21.15.6:0/17635734 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b141070d0 0x7f6b14199790 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:52.438 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.891+0000 7f6b1a8dc700 1 -- 172.21.15.6:0/17635734 >> 172.21.15.6:0/17635734 conn(0x7f6b14100f50 msgr2=0x7f6b14101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:52.439 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.891+0000 7f6b1a8dc700 1 -- 172.21.15.6:0/17635734 shutdown_connections 2024-03-24T12:04:52.439 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:51.891+0000 7f6b1a8dc700 1 -- 172.21.15.6:0/17635734 wait complete. 2024-03-24T12:04:52.441 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-24T12:04:52.441 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout disabled 2024-03-24T12:04:52.448 INFO:teuthology.orchestra.run.smithi006.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-24T12:04:52.448 INFO:teuthology.orchestra.run.smithi006.stdout:systemctl: stdout inactive 2024-03-24T12:04:52.448 INFO:teuthology.orchestra.run.smithi006.stdout:firewalld.service is not enabled 2024-03-24T12:04:52.448 INFO:teuthology.orchestra.run.smithi006.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-24T12:04:52.449 INFO:teuthology.orchestra.run.smithi006.stdout:Ceph Dashboard is now available at: 2024-03-24T12:04:52.449 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:04:52.449 INFO:teuthology.orchestra.run.smithi006.stdout: URL: https://smithi006.front.sepia.ceph.com:8443/ 2024-03-24T12:04:52.450 INFO:teuthology.orchestra.run.smithi006.stdout: User: admin 2024-03-24T12:04:52.450 INFO:teuthology.orchestra.run.smithi006.stdout: Password: n1sqy4bugv 2024-03-24T12:04:52.450 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:04:52.450 INFO:teuthology.orchestra.run.smithi006.stdout:Saving cluster configuration to /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config directory 2024-03-24T12:04:52.450 INFO:teuthology.orchestra.run.smithi006.stdout:Enabling autotune for osd_memory_target 2024-03-24T12:04:53.709 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.925+0000 7f2598c65700 1 Processor -- start 2024-03-24T12:04:53.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.925+0000 7f2598c65700 1 -- start start 2024-03-24T12:04:53.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.926+0000 7f2598c65700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2594107130 0x7f2594107550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:53.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.926+0000 7f2598c65700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2594107b20 con 0x7f2594107130 2024-03-24T12:04:53.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.926+0000 7f259259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2594107130 0x7f2594107550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:53.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.926+0000 7f259259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2594107130 0x7f2594107550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54898/0 (socket says 172.21.15.6:54898) 2024-03-24T12:04:53.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.926+0000 7f259259c700 1 -- 172.21.15.6:0/4176553605 learned_addr learned my addr 172.21.15.6:0/4176553605 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:53.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.926+0000 7f259259c700 1 -- 172.21.15.6:0/4176553605 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2594105670 con 0x7f2594107130 2024-03-24T12:04:53.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.926+0000 7f259259c700 1 --2- 172.21.15.6:0/4176553605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2594107130 0x7f2594107550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f25840098a0 tx=0x7f2584009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a7cebc894f039d server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:53.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.927+0000 7f259159a700 1 -- 172.21.15.6:0/4176553605 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2584004160 con 0x7f2594107130 2024-03-24T12:04:53.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.927+0000 7f259159a700 1 -- 172.21.15.6:0/4176553605 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f25840042c0 con 0x7f2594107130 2024-03-24T12:04:53.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.927+0000 7f259159a700 1 -- 172.21.15.6:0/4176553605 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2584011850 con 0x7f2594107130 2024-03-24T12:04:53.710 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.927+0000 7f2598c65700 1 -- 172.21.15.6:0/4176553605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2594107130 msgr2=0x7f2594107550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:53.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.927+0000 7f2598c65700 1 --2- 172.21.15.6:0/4176553605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2594107130 0x7f2594107550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f25840098a0 tx=0x7f2584009bb0 comp rx=0 tx=0).stop 2024-03-24T12:04:53.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.927+0000 7f2598c65700 1 -- 172.21.15.6:0/4176553605 shutdown_connections 2024-03-24T12:04:53.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.927+0000 7f2598c65700 1 --2- 172.21.15.6:0/4176553605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2594107130 0x7f2594107550 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:53.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.927+0000 7f2598c65700 1 -- 172.21.15.6:0/4176553605 >> 172.21.15.6:0/4176553605 conn(0x7f2594101030 msgr2=0x7f2594103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:53.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.927+0000 7f2598c65700 1 -- 172.21.15.6:0/4176553605 shutdown_connections 2024-03-24T12:04:53.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.927+0000 7f2598c65700 1 -- 172.21.15.6:0/4176553605 wait complete. 2024-03-24T12:04:53.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.928+0000 7f2598c65700 1 Processor -- start 2024-03-24T12:04:53.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.928+0000 7f2598c65700 1 -- start start 2024-03-24T12:04:53.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.928+0000 7f2598c65700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2594107130 0x7f2594199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:53.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.928+0000 7f2598c65700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2594199ca0 con 0x7f2594107130 2024-03-24T12:04:53.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.928+0000 7f259259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2594107130 0x7f2594199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:53.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.928+0000 7f259259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2594107130 0x7f2594199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54910/0 (socket says 172.21.15.6:54910) 2024-03-24T12:04:53.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.928+0000 7f259259c700 1 -- 172.21.15.6:0/651951713 learned_addr learned my addr 172.21.15.6:0/651951713 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:53.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.929+0000 7f259259c700 1 -- 172.21.15.6:0/651951713 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2584009550 con 0x7f2594107130 2024-03-24T12:04:53.711 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.929+0000 7f259259c700 1 --2- 172.21.15.6:0/651951713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2594107130 0x7f2594199760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2584006af0 tx=0x7f2584011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:53.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.929+0000 7f25837fe700 1 -- 172.21.15.6:0/651951713 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25840036a0 con 0x7f2594107130 2024-03-24T12:04:53.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.929+0000 7f25837fe700 1 -- 172.21.15.6:0/651951713 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f258401a430 con 0x7f2594107130 2024-03-24T12:04:53.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.929+0000 7f2598c65700 1 -- 172.21.15.6:0/651951713 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2594199ea0 con 0x7f2594107130 2024-03-24T12:04:53.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.929+0000 7f25837fe700 1 -- 172.21.15.6:0/651951713 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25840193c0 con 0x7f2594107130 2024-03-24T12:04:53.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.929+0000 7f2598c65700 1 -- 172.21.15.6:0/651951713 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259419a340 con 0x7f2594107130 2024-03-24T12:04:53.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.930+0000 7f25837fe700 1 -- 172.21.15.6:0/651951713 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7f2584019520 con 0x7f2594107130 2024-03-24T12:04:53.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.930+0000 7f2598c65700 1 -- 172.21.15.6:0/651951713 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25940632a0 con 0x7f2594107130 2024-03-24T12:04:53.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.930+0000 7f25837fe700 1 --2- 172.21.15.6:0/651951713 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f257c038db0 0x7f257c03b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:53.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.930+0000 7f25837fe700 1 -- 172.21.15.6:0/651951713 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f258404cd50 con 0x7f2594107130 2024-03-24T12:04:53.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.930+0000 7f2591d9b700 1 --2- 172.21.15.6:0/651951713 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f257c038db0 0x7f257c03b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:53.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.930+0000 7f2591d9b700 1 --2- 172.21.15.6:0/651951713 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f257c038db0 0x7f257c03b270 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f258800ad30 tx=0x7f25880093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:53.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:52.935+0000 7f25837fe700 1 -- 172.21.15.6:0/651951713 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f258401a8a0 con 0x7f2594107130 2024-03-24T12:04:53.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:53.136+0000 7f2598c65700 1 -- 172.21.15.6:0/651951713 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f259419cc80 con 0x7f2594107130 2024-03-24T12:04:53.712 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:53.136+0000 7f25837fe700 1 -- 172.21.15.6:0/651951713 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f2584018350 con 0x7f2594107130 2024-03-24T12:04:53.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:53.138+0000 7f2598c65700 1 -- 172.21.15.6:0/651951713 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f257c038db0 msgr2=0x7f257c03b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:53.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:53.138+0000 7f2598c65700 1 --2- 172.21.15.6:0/651951713 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f257c038db0 0x7f257c03b270 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f258800ad30 tx=0x7f25880093f0 comp rx=0 tx=0).stop 2024-03-24T12:04:53.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:53.138+0000 7f2598c65700 1 -- 172.21.15.6:0/651951713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2594107130 msgr2=0x7f2594199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:53.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:53.138+0000 7f2598c65700 1 --2- 172.21.15.6:0/651951713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2594107130 0x7f2594199760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2584006af0 tx=0x7f2584011a50 comp rx=0 tx=0).stop 2024-03-24T12:04:53.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:53.138+0000 7f2598c65700 1 -- 172.21.15.6:0/651951713 shutdown_connections 2024-03-24T12:04:53.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:53.138+0000 7f2598c65700 1 --2- 172.21.15.6:0/651951713 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f257c038db0 0x7f257c03b270 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:53.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:53.138+0000 7f2598c65700 1 --2- 172.21.15.6:0/651951713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2594107130 0x7f2594199760 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:53.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:53.138+0000 7f2598c65700 1 -- 172.21.15.6:0/651951713 >> 172.21.15.6:0/651951713 conn(0x7f2594101030 msgr2=0x7f2594101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:53.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:53.139+0000 7f2598c65700 1 -- 172.21.15.6:0/651951713 shutdown_connections 2024-03-24T12:04:53.713 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:53.139+0000 7f2598c65700 1 -- 172.21.15.6:0/651951713 wait complete. 2024-03-24T12:04:55.041 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.152+0000 7ffa68980700 1 Processor -- start 2024-03-24T12:04:55.042 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.152+0000 7ffa68980700 1 -- start start 2024-03-24T12:04:55.042 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.152+0000 7ffa68980700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa641064f0 0x7ffa641088e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:55.042 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.152+0000 7ffa68980700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa640796e0 con 0x7ffa641064f0 2024-03-24T12:04:55.042 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.152+0000 7ffa6259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa641064f0 0x7ffa641088e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:55.042 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.152+0000 7ffa6259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa641064f0 0x7ffa641088e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54920/0 (socket says 172.21.15.6:54920) 2024-03-24T12:04:55.042 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.152+0000 7ffa6259c700 1 -- 172.21.15.6:0/2149495098 learned_addr learned my addr 172.21.15.6:0/2149495098 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:55.042 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.153+0000 7ffa6259c700 1 -- 172.21.15.6:0/2149495098 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa64108e20 con 0x7ffa641064f0 2024-03-24T12:04:55.042 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.153+0000 7ffa6259c700 1 --2- 172.21.15.6:0/2149495098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa641064f0 0x7ffa641088e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffa540098a0 tx=0x7ffa54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d8d69afc81be07 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:55.042 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.153+0000 7ffa6159a700 1 -- 172.21.15.6:0/2149495098 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa54004160 con 0x7ffa641064f0 2024-03-24T12:04:55.042 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.153+0000 7ffa6159a700 1 -- 172.21.15.6:0/2149495098 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ffa540042c0 con 0x7ffa641064f0 2024-03-24T12:04:55.042 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.153+0000 7ffa6159a700 1 -- 172.21.15.6:0/2149495098 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa54011850 con 0x7ffa641064f0 2024-03-24T12:04:55.043 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.153+0000 7ffa68980700 1 -- 172.21.15.6:0/2149495098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa641064f0 msgr2=0x7ffa641088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:55.043 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.153+0000 7ffa68980700 1 --2- 172.21.15.6:0/2149495098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa641064f0 0x7ffa641088e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffa540098a0 tx=0x7ffa54009bb0 comp rx=0 tx=0).stop 2024-03-24T12:04:55.043 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.154+0000 7ffa68980700 1 -- 172.21.15.6:0/2149495098 shutdown_connections 2024-03-24T12:04:55.043 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.154+0000 7ffa68980700 1 --2- 172.21.15.6:0/2149495098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa641064f0 0x7ffa641088e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:55.043 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.154+0000 7ffa68980700 1 -- 172.21.15.6:0/2149495098 >> 172.21.15.6:0/2149495098 conn(0x7ffa640fff20 msgr2=0x7ffa64102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:55.043 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.154+0000 7ffa68980700 1 -- 172.21.15.6:0/2149495098 shutdown_connections 2024-03-24T12:04:55.043 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.154+0000 7ffa68980700 1 -- 172.21.15.6:0/2149495098 wait complete. 2024-03-24T12:04:55.043 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.154+0000 7ffa68980700 1 Processor -- start 2024-03-24T12:04:55.043 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.154+0000 7ffa68980700 1 -- start start 2024-03-24T12:04:55.043 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.155+0000 7ffa68980700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa641064f0 0x7ffa64199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:55.043 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.155+0000 7ffa68980700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa64199ba0 con 0x7ffa641064f0 2024-03-24T12:04:55.043 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.155+0000 7ffa6259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa641064f0 0x7ffa64199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:55.043 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.155+0000 7ffa6259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa641064f0 0x7ffa64199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:54934/0 (socket says 172.21.15.6:54934) 2024-03-24T12:04:55.043 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.155+0000 7ffa6259c700 1 -- 172.21.15.6:0/1700156944 learned_addr learned my addr 172.21.15.6:0/1700156944 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:55.043 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.155+0000 7ffa6259c700 1 -- 172.21.15.6:0/1700156944 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa54009550 con 0x7ffa641064f0 2024-03-24T12:04:55.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.155+0000 7ffa6259c700 1 --2- 172.21.15.6:0/1700156944 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa641064f0 0x7ffa64199660 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffa54006af0 tx=0x7ffa54011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:55.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.155+0000 7ffa537fe700 1 -- 172.21.15.6:0/1700156944 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa540036a0 con 0x7ffa641064f0 2024-03-24T12:04:55.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.155+0000 7ffa537fe700 1 -- 172.21.15.6:0/1700156944 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ffa5401a430 con 0x7ffa641064f0 2024-03-24T12:04:55.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.155+0000 7ffa68980700 1 -- 172.21.15.6:0/1700156944 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa64199da0 con 0x7ffa641064f0 2024-03-24T12:04:55.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.155+0000 7ffa537fe700 1 -- 172.21.15.6:0/1700156944 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa540193c0 con 0x7ffa641064f0 2024-03-24T12:04:55.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.155+0000 7ffa68980700 1 -- 172.21.15.6:0/1700156944 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa6419a240 con 0x7ffa641064f0 2024-03-24T12:04:55.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.156+0000 7ffa537fe700 1 -- 172.21.15.6:0/1700156944 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7ffa54019680 con 0x7ffa641064f0 2024-03-24T12:04:55.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.156+0000 7ffa68980700 1 -- 172.21.15.6:0/1700156944 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa64109b40 con 0x7ffa641064f0 2024-03-24T12:04:55.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.156+0000 7ffa537fe700 1 --2- 172.21.15.6:0/1700156944 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ffa4c038d50 0x7ffa4c03b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:55.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.156+0000 7ffa537fe700 1 -- 172.21.15.6:0/1700156944 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffa5404d0f0 con 0x7ffa641064f0 2024-03-24T12:04:55.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.156+0000 7ffa61d9b700 1 --2- 172.21.15.6:0/1700156944 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ffa4c038d50 0x7ffa4c03b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:55.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.157+0000 7ffa61d9b700 1 --2- 172.21.15.6:0/1700156944 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ffa4c038d50 0x7ffa4c03b210 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffa58006fd0 tx=0x7ffa58006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:55.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.161+0000 7ffa537fe700 1 -- 172.21.15.6:0/1700156944 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffa54023c20 con 0x7ffa641064f0 2024-03-24T12:04:55.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.475+0000 7ffa68980700 1 -- 172.21.15.6:0/1700156944 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7ffa6419cc20 con 0x7ffa641064f0 2024-03-24T12:04:55.044 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.478+0000 7ffa537fe700 1 -- 172.21.15.6:0/1700156944 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7ffa5400b600 con 0x7ffa641064f0 2024-03-24T12:04:55.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.480+0000 7ffa68980700 1 -- 172.21.15.6:0/1700156944 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ffa4c038d50 msgr2=0x7ffa4c03b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:55.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.480+0000 7ffa68980700 1 --2- 172.21.15.6:0/1700156944 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ffa4c038d50 0x7ffa4c03b210 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffa58006fd0 tx=0x7ffa58006e40 comp rx=0 tx=0).stop 2024-03-24T12:04:55.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.480+0000 7ffa68980700 1 -- 172.21.15.6:0/1700156944 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa641064f0 msgr2=0x7ffa64199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:55.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.480+0000 7ffa68980700 1 --2- 172.21.15.6:0/1700156944 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa641064f0 0x7ffa64199660 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffa54006af0 tx=0x7ffa54011a50 comp rx=0 tx=0).stop 2024-03-24T12:04:55.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.480+0000 7ffa68980700 1 -- 172.21.15.6:0/1700156944 shutdown_connections 2024-03-24T12:04:55.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.480+0000 7ffa68980700 1 --2- 172.21.15.6:0/1700156944 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ffa4c038d50 0x7ffa4c03b210 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:55.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.480+0000 7ffa68980700 1 --2- 172.21.15.6:0/1700156944 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa641064f0 0x7ffa64199660 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:55.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.480+0000 7ffa68980700 1 -- 172.21.15.6:0/1700156944 >> 172.21.15.6:0/1700156944 conn(0x7ffa640fff20 msgr2=0x7ffa64100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:55.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.480+0000 7ffa68980700 1 -- 172.21.15.6:0/1700156944 shutdown_connections 2024-03-24T12:04:55.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr 2024-03-24T12:04:54.480+0000 7ffa68980700 1 -- 172.21.15.6:0/1700156944 wait complete. 2024-03-24T12:04:55.045 INFO:teuthology.orchestra.run.smithi006.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-24T12:04:55.045 INFO:teuthology.orchestra.run.smithi006.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-24T12:04:55.045 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:04:55.045 INFO:teuthology.orchestra.run.smithi006.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-24T12:04:55.045 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:04:55.045 INFO:teuthology.orchestra.run.smithi006.stdout:Or, if you are only running a single cluster on this host: 2024-03-24T12:04:55.046 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:04:55.046 INFO:teuthology.orchestra.run.smithi006.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-24T12:04:55.046 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:04:55.046 INFO:teuthology.orchestra.run.smithi006.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-24T12:04:55.046 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:04:55.046 INFO:teuthology.orchestra.run.smithi006.stdout: ceph telemetry on 2024-03-24T12:04:55.046 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:04:55.046 INFO:teuthology.orchestra.run.smithi006.stdout:For more information see: 2024-03-24T12:04:55.046 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:04:55.046 INFO:teuthology.orchestra.run.smithi006.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-24T12:04:55.046 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:04:55.046 INFO:teuthology.orchestra.run.smithi006.stdout:Bootstrap complete. 2024-03-24T12:04:55.081 INFO:tasks.cephadm:Fetching config... 2024-03-24T12:04:55.081 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-24T12:04:55.081 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-24T12:04:55.097 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-24T12:04:55.097 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-24T12:04:55.097 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-24T12:04:55.152 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-24T12:04:55.152 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-24T12:04:55.152 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-24T12:04:55.218 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-24T12:04:55.219 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-24T12:04:55.219 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-24T12:04:55.274 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-24T12:04:55.274 DEBUG:teuthology.orchestra.run.smithi006:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDayS9KkD/eKNcrAuGYoDxtY9jb4r3LszONEP/OlLyGWWN9Xew86VNTH1uvp5v136RiayVgsGg7fZDbbr1JUXeoGalyoSqWMpHFWpDj0HJZk8bTepLHHpxrD/8N9CXXivK/pN/fb5ULBm8gJtdOxx9humyLOF2sKjvpzWd3X/o955Wsp2hyglbZHrSoij5pc3ly1zzN0rB32LZyWIxqMKWolTwP5MsYopVV/mrsdNreSRCapYn9Yb+15pCJrZqr8OQxrhWtn33C4/wkuk0PbzMaj+3oC2PHlHJlghz6KIUFgZ2+KLPBQJJfyay7Fo62eL/3doziecglRLjYfuvcHZfZ1dNtLJICUIWVi6RUkJJ0V42/C+WPVFJvOd2gE6P1//hCB48EqqQ+hKmlAR5Or07wOCXy8v0neuDTx+rYtrUuTKXS13dgUiWaaFgTu/1OprHlogOU3riC2GK3bp6ebAGaEtTYQnQmNYE1I42slYfDvP2zSmuBzfRRei5Rleefbm8= ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-24T12:04:55.356 INFO:teuthology.orchestra.run.smithi006.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDayS9KkD/eKNcrAuGYoDxtY9jb4r3LszONEP/OlLyGWWN9Xew86VNTH1uvp5v136RiayVgsGg7fZDbbr1JUXeoGalyoSqWMpHFWpDj0HJZk8bTepLHHpxrD/8N9CXXivK/pN/fb5ULBm8gJtdOxx9humyLOF2sKjvpzWd3X/o955Wsp2hyglbZHrSoij5pc3ly1zzN0rB32LZyWIxqMKWolTwP5MsYopVV/mrsdNreSRCapYn9Yb+15pCJrZqr8OQxrhWtn33C4/wkuk0PbzMaj+3oC2PHlHJlghz6KIUFgZ2+KLPBQJJfyay7Fo62eL/3doziecglRLjYfuvcHZfZ1dNtLJICUIWVi6RUkJJ0V42/C+WPVFJvOd2gE6P1//hCB48EqqQ+hKmlAR5Or07wOCXy8v0neuDTx+rYtrUuTKXS13dgUiWaaFgTu/1OprHlogOU3riC2GK3bp6ebAGaEtTYQnQmNYE1I42slYfDvP2zSmuBzfRRei5Rleefbm8= ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715 2024-03-24T12:04:55.370 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDayS9KkD/eKNcrAuGYoDxtY9jb4r3LszONEP/OlLyGWWN9Xew86VNTH1uvp5v136RiayVgsGg7fZDbbr1JUXeoGalyoSqWMpHFWpDj0HJZk8bTepLHHpxrD/8N9CXXivK/pN/fb5ULBm8gJtdOxx9humyLOF2sKjvpzWd3X/o955Wsp2hyglbZHrSoij5pc3ly1zzN0rB32LZyWIxqMKWolTwP5MsYopVV/mrsdNreSRCapYn9Yb+15pCJrZqr8OQxrhWtn33C4/wkuk0PbzMaj+3oC2PHlHJlghz6KIUFgZ2+KLPBQJJfyay7Fo62eL/3doziecglRLjYfuvcHZfZ1dNtLJICUIWVi6RUkJJ0V42/C+WPVFJvOd2gE6P1//hCB48EqqQ+hKmlAR5Or07wOCXy8v0neuDTx+rYtrUuTKXS13dgUiWaaFgTu/1OprHlogOU3riC2GK3bp6ebAGaEtTYQnQmNYE1I42slYfDvP2zSmuBzfRRei5Rleefbm8= ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-24T12:04:55.418 INFO:teuthology.orchestra.run.smithi090.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDayS9KkD/eKNcrAuGYoDxtY9jb4r3LszONEP/OlLyGWWN9Xew86VNTH1uvp5v136RiayVgsGg7fZDbbr1JUXeoGalyoSqWMpHFWpDj0HJZk8bTepLHHpxrD/8N9CXXivK/pN/fb5ULBm8gJtdOxx9humyLOF2sKjvpzWd3X/o955Wsp2hyglbZHrSoij5pc3ly1zzN0rB32LZyWIxqMKWolTwP5MsYopVV/mrsdNreSRCapYn9Yb+15pCJrZqr8OQxrhWtn33C4/wkuk0PbzMaj+3oC2PHlHJlghz6KIUFgZ2+KLPBQJJfyay7Fo62eL/3doziecglRLjYfuvcHZfZ1dNtLJICUIWVi6RUkJJ0V42/C+WPVFJvOd2gE6P1//hCB48EqqQ+hKmlAR5Or07wOCXy8v0neuDTx+rYtrUuTKXS13dgUiWaaFgTu/1OprHlogOU3riC2GK3bp6ebAGaEtTYQnQmNYE1I42slYfDvP2zSmuBzfRRei5Rleefbm8= ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715 2024-03-24T12:04:55.433 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-24T12:04:55.670 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:04:55.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:55 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1700156944' entity='client.admin' 2024-03-24T12:04:58.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.112+0000 7f56bcf95700 1 -- 172.21.15.6:0/2127546859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56b81016b0 msgr2=0x7f56b8101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:58.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.112+0000 7f56bcf95700 1 --2- 172.21.15.6:0/2127546859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56b81016b0 0x7f56b8101ad0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009910 tx=0x7f56ac009c20 comp rx=0 tx=0).stop 2024-03-24T12:04:58.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.112+0000 7f56bcf95700 1 -- 172.21.15.6:0/2127546859 shutdown_connections 2024-03-24T12:04:58.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.112+0000 7f56bcf95700 1 --2- 172.21.15.6:0/2127546859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56b81016b0 0x7f56b8101ad0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:58.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.112+0000 7f56bcf95700 1 -- 172.21.15.6:0/2127546859 >> 172.21.15.6:0/2127546859 conn(0x7f56b80fcc50 msgr2=0x7f56b80ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:58.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.112+0000 7f56bcf95700 1 -- 172.21.15.6:0/2127546859 shutdown_connections 2024-03-24T12:04:58.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.112+0000 7f56bcf95700 1 -- 172.21.15.6:0/2127546859 wait complete. 2024-03-24T12:04:58.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.113+0000 7f56bcf95700 1 Processor -- start 2024-03-24T12:04:58.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.113+0000 7f56bcf95700 1 -- start start 2024-03-24T12:04:58.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.113+0000 7f56bcf95700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56b81016b0 0x7f56b8196120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:58.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.113+0000 7f56bcf95700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b8196660 con 0x7f56b81016b0 2024-03-24T12:04:58.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.113+0000 7f56b659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56b81016b0 0x7f56b8196120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:58.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.113+0000 7f56b659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56b81016b0 0x7f56b8196120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39278/0 (socket says 172.21.15.6:39278) 2024-03-24T12:04:58.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.113+0000 7f56b659c700 1 -- 172.21.15.6:0/2842565496 learned_addr learned my addr 172.21.15.6:0/2842565496 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:04:58.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.114+0000 7f56b659c700 1 -- 172.21.15.6:0/2842565496 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ac0095f0 con 0x7f56b81016b0 2024-03-24T12:04:58.114 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.114+0000 7f56b659c700 1 --2- 172.21.15.6:0/2842565496 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56b81016b0 0x7f56b8196120 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f56ac005660 tx=0x7f56ac005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:58.115 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.114+0000 7f56a77fe700 1 -- 172.21.15.6:0/2842565496 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56ac00b960 con 0x7f56b81016b0 2024-03-24T12:04:58.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.114+0000 7f56a77fe700 1 -- 172.21.15.6:0/2842565496 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f56ac00f460 con 0x7f56b81016b0 2024-03-24T12:04:58.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.114+0000 7f56bcf95700 1 -- 172.21.15.6:0/2842565496 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b8196860 con 0x7f56b81016b0 2024-03-24T12:04:58.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.115+0000 7f56a77fe700 1 -- 172.21.15.6:0/2842565496 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56ac017440 con 0x7f56b81016b0 2024-03-24T12:04:58.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.115+0000 7f56bcf95700 1 -- 172.21.15.6:0/2842565496 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b8196d00 con 0x7f56b81016b0 2024-03-24T12:04:58.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.115+0000 7f56a77fe700 1 -- 172.21.15.6:0/2842565496 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7f56ac0176b0 con 0x7f56b81016b0 2024-03-24T12:04:58.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.115+0000 7f56a77fe700 1 --2- 172.21.15.6:0/2842565496 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f56a0038db0 0x7f56a003b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:04:58.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.115+0000 7f56a77fe700 1 -- 172.21.15.6:0/2842565496 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f56ac00f5d0 con 0x7f56b81016b0 2024-03-24T12:04:58.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.115+0000 7f56b5d9b700 1 --2- 172.21.15.6:0/2842565496 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f56a0038db0 0x7f56a003b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:04:58.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.116+0000 7f56b5d9b700 1 --2- 172.21.15.6:0/2842565496 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f56a0038db0 0x7f56a003b270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f56a8006fd0 tx=0x7f56a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:04:58.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.116+0000 7f56bcf95700 1 -- 172.21.15.6:0/2842565496 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b818ff30 con 0x7f56b81016b0 2024-03-24T12:04:58.120 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.121+0000 7f56a77fe700 1 -- 172.21.15.6:0/2842565496 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f56ac027080 con 0x7f56b81016b0 2024-03-24T12:04:58.324 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.324+0000 7f56bcf95700 1 -- 172.21.15.6:0/2842565496 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f56b80008d0 con 0x7f56b81016b0 2024-03-24T12:04:58.324 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:58 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:04:58.324 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:58 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:04:58.324 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:58 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi006", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:04:58.324 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:58 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:04:58.325 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:58 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:04:58.325 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:58 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:04:58.328 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.329+0000 7f56a77fe700 1 -- 172.21.15.6:0/2842565496 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f56ac054060 con 0x7f56b81016b0 2024-03-24T12:04:58.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.331+0000 7f56bcf95700 1 -- 172.21.15.6:0/2842565496 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f56a0038db0 msgr2=0x7f56a003b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:58.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.331+0000 7f56bcf95700 1 --2- 172.21.15.6:0/2842565496 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f56a0038db0 0x7f56a003b270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f56a8006fd0 tx=0x7f56a8006e40 comp rx=0 tx=0).stop 2024-03-24T12:04:58.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.331+0000 7f56bcf95700 1 -- 172.21.15.6:0/2842565496 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56b81016b0 msgr2=0x7f56b8196120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:04:58.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.331+0000 7f56bcf95700 1 --2- 172.21.15.6:0/2842565496 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56b81016b0 0x7f56b8196120 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f56ac005660 tx=0x7f56ac005740 comp rx=0 tx=0).stop 2024-03-24T12:04:58.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.331+0000 7f56bcf95700 1 -- 172.21.15.6:0/2842565496 shutdown_connections 2024-03-24T12:04:58.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.331+0000 7f56bcf95700 1 --2- 172.21.15.6:0/2842565496 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f56a0038db0 0x7f56a003b270 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:58.331 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.331+0000 7f56bcf95700 1 --2- 172.21.15.6:0/2842565496 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56b81016b0 0x7f56b8196120 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:04:58.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.331+0000 7f56bcf95700 1 -- 172.21.15.6:0/2842565496 >> 172.21.15.6:0/2842565496 conn(0x7f56b80fcc50 msgr2=0x7f56b80fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:04:58.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.332+0000 7f56bcf95700 1 -- 172.21.15.6:0/2842565496 shutdown_connections 2024-03-24T12:04:58.332 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:04:58.332+0000 7f56bcf95700 1 -- 172.21.15.6:0/2842565496 wait complete. 2024-03-24T12:04:58.985 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-24T12:04:58.986 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-24T12:04:59.572 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:59 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2842565496' entity='client.admin' 2024-03-24T12:04:59.572 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:59 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:04:59.572 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:04:59 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:04:59.572 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:05:01.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.062+0000 7fb113591700 1 -- 172.21.15.6:0/204858823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb10c1016c0 msgr2=0x7fb10c101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:01.088 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.062+0000 7fb113591700 1 --2- 172.21.15.6:0/204858823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb10c1016c0 0x7fb10c101ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb108009910 tx=0x7fb108009c20 comp rx=0 tx=0).stop 2024-03-24T12:05:01.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.062+0000 7fb113591700 1 -- 172.21.15.6:0/204858823 shutdown_connections 2024-03-24T12:05:01.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.062+0000 7fb113591700 1 --2- 172.21.15.6:0/204858823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb10c1016c0 0x7fb10c101ae0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:01.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.062+0000 7fb113591700 1 -- 172.21.15.6:0/204858823 >> 172.21.15.6:0/204858823 conn(0x7fb10c0fcc60 msgr2=0x7fb10c0ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:05:01.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.062+0000 7fb113591700 1 -- 172.21.15.6:0/204858823 shutdown_connections 2024-03-24T12:05:01.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.063+0000 7fb113591700 1 -- 172.21.15.6:0/204858823 wait complete. 2024-03-24T12:05:01.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.063+0000 7fb113591700 1 Processor -- start 2024-03-24T12:05:01.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.063+0000 7fb113591700 1 -- start start 2024-03-24T12:05:01.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.063+0000 7fb113591700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb10c1016c0 0x7fb10c196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:01.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.063+0000 7fb113591700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c1966b0 con 0x7fb10c1016c0 2024-03-24T12:05:01.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.063+0000 7fb11132d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb10c1016c0 0x7fb10c196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:01.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.063+0000 7fb11132d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb10c1016c0 0x7fb10c196170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39282/0 (socket says 172.21.15.6:39282) 2024-03-24T12:05:01.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.064+0000 7fb11132d700 1 -- 172.21.15.6:0/2961242028 learned_addr learned my addr 172.21.15.6:0/2961242028 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:05:01.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.064+0000 7fb11132d700 1 -- 172.21.15.6:0/2961242028 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1080095f0 con 0x7fb10c1016c0 2024-03-24T12:05:01.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.064+0000 7fb11132d700 1 --2- 172.21.15.6:0/2961242028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb10c1016c0 0x7fb10c196170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb108005660 tx=0x7fb108005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:05:01.089 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.064+0000 7fb1027fc700 1 -- 172.21.15.6:0/2961242028 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb10800b960 con 0x7fb10c1016c0 2024-03-24T12:05:01.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.064+0000 7fb1027fc700 1 -- 172.21.15.6:0/2961242028 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fb10800f460 con 0x7fb10c1016c0 2024-03-24T12:05:01.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.064+0000 7fb113591700 1 -- 172.21.15.6:0/2961242028 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10c1968b0 con 0x7fb10c1016c0 2024-03-24T12:05:01.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.064+0000 7fb1027fc700 1 -- 172.21.15.6:0/2961242028 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb108017440 con 0x7fb10c1016c0 2024-03-24T12:05:01.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.064+0000 7fb113591700 1 -- 172.21.15.6:0/2961242028 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10c196d50 con 0x7fb10c1016c0 2024-03-24T12:05:01.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.065+0000 7fb1027fc700 1 -- 172.21.15.6:0/2961242028 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7fb10800bac0 con 0x7fb10c1016c0 2024-03-24T12:05:01.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.065+0000 7fb113591700 1 -- 172.21.15.6:0/2961242028 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10c18ff40 con 0x7fb10c1016c0 2024-03-24T12:05:01.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.065+0000 7fb1027fc700 1 --2- 172.21.15.6:0/2961242028 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fb0f8038d50 0x7fb0f803b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:01.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.065+0000 7fb1027fc700 1 -- 172.21.15.6:0/2961242028 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb10804cf70 con 0x7fb10c1016c0 2024-03-24T12:05:01.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.065+0000 7fb110b2c700 1 --2- 172.21.15.6:0/2961242028 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fb0f8038d50 0x7fb0f803b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:01.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.066+0000 7fb110b2c700 1 --2- 172.21.15.6:0/2961242028 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fb0f8038d50 0x7fb0f803b210 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc006fd0 tx=0x7fb0fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:05:01.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.070+0000 7fb1027fc700 1 -- 172.21.15.6:0/2961242028 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb10800f5d0 con 0x7fb10c1016c0 2024-03-24T12:05:01.276 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.277+0000 7fb113591700 1 -- 172.21.15.6:0/2961242028 --> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fb10c106bf0 con 0x7fb0f8038d50 2024-03-24T12:05:01.279 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.280+0000 7fb1027fc700 1 -- 172.21.15.6:0/2961242028 <== mgr.14152 v2:172.21.15.6:6800/3847865179 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb10c106bf0 con 0x7fb0f8038d50 2024-03-24T12:05:01.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.282+0000 7fb113591700 1 -- 172.21.15.6:0/2961242028 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fb0f8038d50 msgr2=0x7fb0f803b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:01.281 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.282+0000 7fb113591700 1 --2- 172.21.15.6:0/2961242028 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fb0f8038d50 0x7fb0f803b210 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc006fd0 tx=0x7fb0fc006e40 comp rx=0 tx=0).stop 2024-03-24T12:05:01.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.282+0000 7fb113591700 1 -- 172.21.15.6:0/2961242028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb10c1016c0 msgr2=0x7fb10c196170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:01.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.282+0000 7fb113591700 1 --2- 172.21.15.6:0/2961242028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb10c1016c0 0x7fb10c196170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb108005660 tx=0x7fb108005740 comp rx=0 tx=0).stop 2024-03-24T12:05:01.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.282+0000 7fb113591700 1 -- 172.21.15.6:0/2961242028 shutdown_connections 2024-03-24T12:05:01.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.282+0000 7fb113591700 1 --2- 172.21.15.6:0/2961242028 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fb0f8038d50 0x7fb0f803b210 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:01.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.282+0000 7fb113591700 1 --2- 172.21.15.6:0/2961242028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb10c1016c0 0x7fb10c196170 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:01.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.282+0000 7fb113591700 1 -- 172.21.15.6:0/2961242028 >> 172.21.15.6:0/2961242028 conn(0x7fb10c0fcc60 msgr2=0x7fb10c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:05:01.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.282+0000 7fb113591700 1 -- 172.21.15.6:0/2961242028 shutdown_connections 2024-03-24T12:05:01.282 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:01.282+0000 7fb113591700 1 -- 172.21.15.6:0/2961242028 wait complete. 2024-03-24T12:05:02.035 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi090 2024-03-24T12:05:02.035 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-24T12:05:02.035 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/etc/ceph/ceph.conf 2024-03-24T12:05:02.053 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-24T12:05:02.053 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-24T12:05:02.110 INFO:tasks.cephadm:Adding host smithi090 to orchestrator... 2024-03-24T12:05:02.111 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph orch host add smithi090 2024-03-24T12:05:02.351 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:05:02.562 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:02 smithi006 ceph-mon[30108]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:05:02.562 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:02 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:02.563 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:02 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:05:02.563 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:02 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:02.563 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:02 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:05:02.563 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:02 smithi006 ceph-mon[30108]: Updating smithi006:/etc/ceph/ceph.conf 2024-03-24T12:05:02.563 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:02 smithi006 ceph-mon[30108]: Updating smithi006:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T12:05:02.563 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:02 smithi006 ceph-mon[30108]: Updating smithi006:/etc/ceph/ceph.client.admin.keyring 2024-03-24T12:05:02.563 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:02 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:02.563 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:02 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:02.563 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:02 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:03.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:03 smithi006 ceph-mon[30108]: Updating smithi006:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T12:05:03.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.869+0000 7efdb0ef8700 1 -- 172.21.15.6:0/132015481 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efdac102860 msgr2=0x7efdac102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:03.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.869+0000 7efdb0ef8700 1 --2- 172.21.15.6:0/132015481 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efdac102860 0x7efdac102c40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009910 tx=0x7efd9c009c20 comp rx=0 tx=0).stop 2024-03-24T12:05:03.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.869+0000 7efdb0ef8700 1 -- 172.21.15.6:0/132015481 shutdown_connections 2024-03-24T12:05:03.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.869+0000 7efdb0ef8700 1 --2- 172.21.15.6:0/132015481 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efdac102860 0x7efdac102c40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:03.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.869+0000 7efdb0ef8700 1 -- 172.21.15.6:0/132015481 >> 172.21.15.6:0/132015481 conn(0x7efdac0771b0 msgr2=0x7efdac0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:05:03.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.869+0000 7efdb0ef8700 1 -- 172.21.15.6:0/132015481 shutdown_connections 2024-03-24T12:05:03.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.869+0000 7efdb0ef8700 1 -- 172.21.15.6:0/132015481 wait complete. 2024-03-24T12:05:03.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.870+0000 7efdb0ef8700 1 Processor -- start 2024-03-24T12:05:03.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.870+0000 7efdb0ef8700 1 -- start start 2024-03-24T12:05:03.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.870+0000 7efdb0ef8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efdac102860 0x7efdac1004c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:03.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.870+0000 7efdb0ef8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac1023b0 con 0x7efdac102860 2024-03-24T12:05:03.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.870+0000 7efdaa59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efdac102860 0x7efdac1004c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:03.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.870+0000 7efdaa59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efdac102860 0x7efdac1004c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:39302/0 (socket says 172.21.15.6:39302) 2024-03-24T12:05:03.871 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.870+0000 7efdaa59c700 1 -- 172.21.15.6:0/2743862799 learned_addr learned my addr 172.21.15.6:0/2743862799 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:05:03.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.871+0000 7efdaa59c700 1 -- 172.21.15.6:0/2743862799 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9c0095f0 con 0x7efdac102860 2024-03-24T12:05:03.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.871+0000 7efdaa59c700 1 --2- 172.21.15.6:0/2743862799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efdac102860 0x7efdac1004c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efd9c006010 tx=0x7efd9c005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:05:03.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.871+0000 7efd9b7fe700 1 -- 172.21.15.6:0/2743862799 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd9c01d070 con 0x7efdac102860 2024-03-24T12:05:03.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.871+0000 7efdb0ef8700 1 -- 172.21.15.6:0/2743862799 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdac100a00 con 0x7efdac102860 2024-03-24T12:05:03.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.871+0000 7efd9b7fe700 1 -- 172.21.15.6:0/2743862799 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7efd9c00b750 con 0x7efdac102860 2024-03-24T12:05:03.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.871+0000 7efd9b7fe700 1 -- 172.21.15.6:0/2743862799 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd9c0037f0 con 0x7efdac102860 2024-03-24T12:05:03.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.871+0000 7efdb0ef8700 1 -- 172.21.15.6:0/2743862799 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdac100e20 con 0x7efdac102860 2024-03-24T12:05:03.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.872+0000 7efd9b7fe700 1 -- 172.21.15.6:0/2743862799 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7efd9c003a70 con 0x7efdac102860 2024-03-24T12:05:03.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.872+0000 7efdb0ef8700 1 -- 172.21.15.6:0/2743862799 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdac108660 con 0x7efdac102860 2024-03-24T12:05:03.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.872+0000 7efd9b7fe700 1 --2- 172.21.15.6:0/2743862799 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7efd94038d50 0x7efd9403b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:03.873 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.872+0000 7efd9b7fe700 1 -- 172.21.15.6:0/2743862799 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efd9c04ca80 con 0x7efdac102860 2024-03-24T12:05:03.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.872+0000 7efda9d9b700 1 --2- 172.21.15.6:0/2743862799 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7efd94038d50 0x7efd9403b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:03.874 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.873+0000 7efda9d9b700 1 --2- 172.21.15.6:0/2743862799 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7efd94038d50 0x7efd9403b210 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efda0006fd0 tx=0x7efda0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:05:03.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:03.881+0000 7efd9b7fe700 1 -- 172.21.15.6:0/2743862799 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efd9c029360 con 0x7efdac102860 2024-03-24T12:05:04.090 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:04.088+0000 7efdb0ef8700 1 -- 172.21.15.6:0/2743862799 --> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi090", "target": ["mon-mgr", ""]}) v1 -- 0x7efdac101a60 con 0x7efd94038d50 2024-03-24T12:05:05.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:04 smithi006 ceph-mon[30108]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi090", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:05:05.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:04 smithi006 ceph-mon[30108]: Deploying cephadm binary to smithi090 2024-03-24T12:05:06.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:05 smithi006 ceph-mon[30108]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:08.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:07 smithi006 ceph-mon[30108]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:08.484 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:08.484+0000 7efd9b7fe700 1 -- 172.21.15.6:0/2743862799 <== mgr.14152 v2:172.21.15.6:6800/3847865179 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7efdac101a60 con 0x7efd94038d50 2024-03-24T12:05:08.485 INFO:teuthology.orchestra.run.smithi006.stdout:Added host 'smithi090' with addr '172.21.15.90' 2024-03-24T12:05:08.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:08.487+0000 7efdb0ef8700 1 -- 172.21.15.6:0/2743862799 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7efd94038d50 msgr2=0x7efd9403b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:08.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:08.487+0000 7efdb0ef8700 1 --2- 172.21.15.6:0/2743862799 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7efd94038d50 0x7efd9403b210 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efda0006fd0 tx=0x7efda0006e40 comp rx=0 tx=0).stop 2024-03-24T12:05:08.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:08.487+0000 7efdb0ef8700 1 -- 172.21.15.6:0/2743862799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efdac102860 msgr2=0x7efdac1004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:08.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:08.487+0000 7efdb0ef8700 1 --2- 172.21.15.6:0/2743862799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efdac102860 0x7efdac1004c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efd9c006010 tx=0x7efd9c005710 comp rx=0 tx=0).stop 2024-03-24T12:05:08.487 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:08.487+0000 7efdb0ef8700 1 -- 172.21.15.6:0/2743862799 shutdown_connections 2024-03-24T12:05:08.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:08.487+0000 7efdb0ef8700 1 --2- 172.21.15.6:0/2743862799 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7efd94038d50 0x7efd9403b210 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:08.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:08.487+0000 7efdb0ef8700 1 --2- 172.21.15.6:0/2743862799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efdac102860 0x7efdac1004c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:08.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:08.487+0000 7efdb0ef8700 1 -- 172.21.15.6:0/2743862799 >> 172.21.15.6:0/2743862799 conn(0x7efdac0771b0 msgr2=0x7efdac0fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:05:08.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:08.487+0000 7efdb0ef8700 1 -- 172.21.15.6:0/2743862799 shutdown_connections 2024-03-24T12:05:08.488 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:08.487+0000 7efdb0ef8700 1 -- 172.21.15.6:0/2743862799 wait complete. 2024-03-24T12:05:09.159 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-24T12:05:09.406 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:05:09.794 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:09 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:09.794 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:09 smithi006 ceph-mon[30108]: Added host smithi090 2024-03-24T12:05:09.794 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:09 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:05:09.794 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:09 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:10.850 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:10 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:10.851 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:10 smithi006 ceph-mon[30108]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:10.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.862+0000 7f331dc2b700 1 -- 172.21.15.6:0/2142979426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3318106620 msgr2=0x7f3318106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:10.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.862+0000 7f331dc2b700 1 --2- 172.21.15.6:0/2142979426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3318106620 0x7f3318106a00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3308009910 tx=0x7f3308009c20 comp rx=0 tx=0).stop 2024-03-24T12:05:10.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.862+0000 7f331dc2b700 1 -- 172.21.15.6:0/2142979426 shutdown_connections 2024-03-24T12:05:10.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.862+0000 7f331dc2b700 1 --2- 172.21.15.6:0/2142979426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3318106620 0x7f3318106a00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:10.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.862+0000 7f331dc2b700 1 -- 172.21.15.6:0/2142979426 >> 172.21.15.6:0/2142979426 conn(0x7f33180fc040 msgr2=0x7f33180fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:05:10.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.862+0000 7f331dc2b700 1 -- 172.21.15.6:0/2142979426 shutdown_connections 2024-03-24T12:05:10.863 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.862+0000 7f331dc2b700 1 -- 172.21.15.6:0/2142979426 wait complete. 2024-03-24T12:05:10.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.863+0000 7f331dc2b700 1 Processor -- start 2024-03-24T12:05:10.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.863+0000 7f331dc2b700 1 -- start start 2024-03-24T12:05:10.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.863+0000 7f331dc2b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3318106620 0x7f3318195320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:10.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.863+0000 7f331dc2b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3318195860 con 0x7f3318106620 2024-03-24T12:05:10.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.864+0000 7f33177fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3318106620 0x7f3318195320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:10.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.864+0000 7f33177fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3318106620 0x7f3318195320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51582/0 (socket says 172.21.15.6:51582) 2024-03-24T12:05:10.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.864+0000 7f33177fe700 1 -- 172.21.15.6:0/2279731823 learned_addr learned my addr 172.21.15.6:0/2279731823 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:05:10.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.864+0000 7f33177fe700 1 -- 172.21.15.6:0/2279731823 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33080095f0 con 0x7f3318106620 2024-03-24T12:05:10.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.864+0000 7f33177fe700 1 --2- 172.21.15.6:0/2279731823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3318106620 0x7f3318195320 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3308006010 tx=0x7f3308005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:05:10.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.864+0000 7f3314ff9700 1 -- 172.21.15.6:0/2279731823 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f330801f070 con 0x7f3318106620 2024-03-24T12:05:10.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.864+0000 7f3314ff9700 1 -- 172.21.15.6:0/2279731823 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f330800b750 con 0x7f3318106620 2024-03-24T12:05:10.865 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.864+0000 7f331dc2b700 1 -- 172.21.15.6:0/2279731823 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3318195a60 con 0x7f3318106620 2024-03-24T12:05:10.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.864+0000 7f331dc2b700 1 -- 172.21.15.6:0/2279731823 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3318195f60 con 0x7f3318106620 2024-03-24T12:05:10.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.864+0000 7f3314ff9700 1 -- 172.21.15.6:0/2279731823 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3308003810 con 0x7f3318106620 2024-03-24T12:05:10.866 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.865+0000 7f3314ff9700 1 -- 172.21.15.6:0/2279731823 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7f3308003a00 con 0x7f3318106620 2024-03-24T12:05:10.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.865+0000 7f331dc2b700 1 -- 172.21.15.6:0/2279731823 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33180632a0 con 0x7f3318106620 2024-03-24T12:05:10.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.865+0000 7f3314ff9700 1 --2- 172.21.15.6:0/2279731823 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f3300038db0 0x7f330003b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:10.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.865+0000 7f3314ff9700 1 -- 172.21.15.6:0/2279731823 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f330804ee80 con 0x7f3318106620 2024-03-24T12:05:10.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.865+0000 7f3316ffd700 1 --2- 172.21.15.6:0/2279731823 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f3300038db0 0x7f330003b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:10.867 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.866+0000 7f3316ffd700 1 --2- 172.21.15.6:0/2279731823 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f3300038db0 0x7f330003b270 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f330c006fd0 tx=0x7f330c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:05:10.870 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:10.870+0000 7f3314ff9700 1 -- 172.21.15.6:0/2279731823 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f330802b770 con 0x7f3318106620 2024-03-24T12:05:11.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:11.081+0000 7f331dc2b700 1 -- 172.21.15.6:0/2279731823 --> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3318109db0 con 0x7f3300038db0 2024-03-24T12:05:11.082 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:11.082+0000 7f3314ff9700 1 -- 172.21.15.6:0/2279731823 <== mgr.14152 v2:172.21.15.6:6800/3847865179 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+158 (secure 0 0 0) 0x7f3318109db0 con 0x7f3300038db0 2024-03-24T12:05:11.082 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:05:11.083 INFO:teuthology.orchestra.run.smithi006.stdout:[{"addr": "172.21.15.6", "hostname": "smithi006", "labels": [], "status": ""}, {"addr": "172.21.15.90", "hostname": "smithi090", "labels": [], "status": ""}] 2024-03-24T12:05:11.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:11.084+0000 7f331dc2b700 1 -- 172.21.15.6:0/2279731823 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f3300038db0 msgr2=0x7f330003b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:11.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:11.084+0000 7f331dc2b700 1 --2- 172.21.15.6:0/2279731823 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f3300038db0 0x7f330003b270 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f330c006fd0 tx=0x7f330c006e40 comp rx=0 tx=0).stop 2024-03-24T12:05:11.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:11.084+0000 7f331dc2b700 1 -- 172.21.15.6:0/2279731823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3318106620 msgr2=0x7f3318195320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:11.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:11.084+0000 7f331dc2b700 1 --2- 172.21.15.6:0/2279731823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3318106620 0x7f3318195320 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3308006010 tx=0x7f3308005710 comp rx=0 tx=0).stop 2024-03-24T12:05:11.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:11.084+0000 7f331dc2b700 1 -- 172.21.15.6:0/2279731823 shutdown_connections 2024-03-24T12:05:11.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:11.084+0000 7f331dc2b700 1 --2- 172.21.15.6:0/2279731823 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f3300038db0 0x7f330003b270 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:11.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:11.084+0000 7f331dc2b700 1 --2- 172.21.15.6:0/2279731823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3318106620 0x7f3318195320 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:11.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:11.084+0000 7f331dc2b700 1 -- 172.21.15.6:0/2279731823 >> 172.21.15.6:0/2279731823 conn(0x7f33180fc040 msgr2=0x7f33180fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:05:11.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:11.084+0000 7f331dc2b700 1 -- 172.21.15.6:0/2279731823 shutdown_connections 2024-03-24T12:05:11.085 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:11.084+0000 7f331dc2b700 1 -- 172.21.15.6:0/2279731823 wait complete. 2024-03-24T12:05:11.657 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-24T12:05:11.658 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph osd crush tunables default 2024-03-24T12:05:11.897 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:05:12.324 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:12 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:12.324 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:12 smithi006 ceph-mon[30108]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T12:05:12.324 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:12 smithi006 ceph-mon[30108]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:13.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.434+0000 7eff6c702700 1 -- 172.21.15.6:0/1808730646 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff640fff30 msgr2=0x7eff64100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:13.435 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.434+0000 7eff6c702700 1 --2- 172.21.15.6:0/1808730646 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff640fff30 0x7eff64100310 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7eff54009910 tx=0x7eff54009c20 comp rx=0 tx=0).stop 2024-03-24T12:05:13.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.434+0000 7eff6c702700 1 -- 172.21.15.6:0/1808730646 shutdown_connections 2024-03-24T12:05:13.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.434+0000 7eff6c702700 1 --2- 172.21.15.6:0/1808730646 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff640fff30 0x7eff64100310 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:13.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.434+0000 7eff6c702700 1 -- 172.21.15.6:0/1808730646 >> 172.21.15.6:0/1808730646 conn(0x7eff640fb7c0 msgr2=0x7eff640fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:05:13.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.434+0000 7eff6c702700 1 -- 172.21.15.6:0/1808730646 shutdown_connections 2024-03-24T12:05:13.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.434+0000 7eff6c702700 1 -- 172.21.15.6:0/1808730646 wait complete. 2024-03-24T12:05:13.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.435+0000 7eff6c702700 1 Processor -- start 2024-03-24T12:05:13.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.435+0000 7eff6c702700 1 -- start start 2024-03-24T12:05:13.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.435+0000 7eff6c702700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff640fff30 0x7eff640fbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:13.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.435+0000 7eff6c702700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff640fdb60 con 0x7eff640fff30 2024-03-24T12:05:13.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.435+0000 7eff6a49e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff640fff30 0x7eff640fbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:13.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.435+0000 7eff6a49e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff640fff30 0x7eff640fbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:51600/0 (socket says 172.21.15.6:51600) 2024-03-24T12:05:13.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.435+0000 7eff6a49e700 1 -- 172.21.15.6:0/71740740 learned_addr learned my addr 172.21.15.6:0/71740740 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:05:13.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.435+0000 7eff6a49e700 1 -- 172.21.15.6:0/71740740 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff540095f0 con 0x7eff640fff30 2024-03-24T12:05:13.436 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.435+0000 7eff6a49e700 1 --2- 172.21.15.6:0/71740740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff640fff30 0x7eff640fbc70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7eff54005660 tx=0x7eff54005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:05:13.437 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.435+0000 7eff5b7fe700 1 -- 172.21.15.6:0/71740740 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff5401d070 con 0x7eff640fff30 2024-03-24T12:05:13.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.435+0000 7eff5b7fe700 1 -- 172.21.15.6:0/71740740 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7eff5400b750 con 0x7eff640fff30 2024-03-24T12:05:13.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.435+0000 7eff6c702700 1 -- 172.21.15.6:0/71740740 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff640fc1b0 con 0x7eff640fff30 2024-03-24T12:05:13.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.435+0000 7eff5b7fe700 1 -- 172.21.15.6:0/71740740 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff540037b0 con 0x7eff640fff30 2024-03-24T12:05:13.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.436+0000 7eff6c702700 1 -- 172.21.15.6:0/71740740 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff640fc5d0 con 0x7eff640fff30 2024-03-24T12:05:13.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.437+0000 7eff5b7fe700 1 -- 172.21.15.6:0/71740740 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7eff54004390 con 0x7eff640fff30 2024-03-24T12:05:13.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.437+0000 7eff6c702700 1 -- 172.21.15.6:0/71740740 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff640632a0 con 0x7eff640fff30 2024-03-24T12:05:13.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.437+0000 7eff5b7fe700 1 --2- 172.21.15.6:0/71740740 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7eff50038d50 0x7eff5003b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:13.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.437+0000 7eff5b7fe700 1 -- 172.21.15.6:0/71740740 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7eff5404ca50 con 0x7eff640fff30 2024-03-24T12:05:13.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.437+0000 7eff69c9d700 1 --2- 172.21.15.6:0/71740740 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7eff50038d50 0x7eff5003b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:13.438 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.437+0000 7eff69c9d700 1 --2- 172.21.15.6:0/71740740 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7eff50038d50 0x7eff5003b210 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7eff60006fd0 tx=0x7eff60006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:05:13.442 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.442+0000 7eff5b7fe700 1 -- 172.21.15.6:0/71740740 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7eff54003df0 con 0x7eff640fff30 2024-03-24T12:05:13.650 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.650+0000 7eff6c702700 1 -- 172.21.15.6:0/71740740 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7eff640fd1a0 con 0x7eff640fff30 2024-03-24T12:05:13.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.701+0000 7eff5b7fe700 1 -- 172.21.15.6:0/71740740 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7eff640fd1a0 con 0x7eff640fff30 2024-03-24T12:05:13.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.703+0000 7eff6c702700 1 -- 172.21.15.6:0/71740740 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7eff50038d50 msgr2=0x7eff5003b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:13.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.703+0000 7eff6c702700 1 --2- 172.21.15.6:0/71740740 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7eff50038d50 0x7eff5003b210 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7eff60006fd0 tx=0x7eff60006e40 comp rx=0 tx=0).stop 2024-03-24T12:05:13.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.703+0000 7eff6c702700 1 -- 172.21.15.6:0/71740740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff640fff30 msgr2=0x7eff640fbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:13.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.703+0000 7eff6c702700 1 --2- 172.21.15.6:0/71740740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff640fff30 0x7eff640fbc70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7eff54005660 tx=0x7eff54005740 comp rx=0 tx=0).stop 2024-03-24T12:05:13.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.703+0000 7eff6c702700 1 -- 172.21.15.6:0/71740740 shutdown_connections 2024-03-24T12:05:13.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.703+0000 7eff6c702700 1 --2- 172.21.15.6:0/71740740 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7eff50038d50 0x7eff5003b210 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:13.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.703+0000 7eff6c702700 1 --2- 172.21.15.6:0/71740740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff640fff30 0x7eff640fbc70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:13.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.703+0000 7eff6c702700 1 -- 172.21.15.6:0/71740740 >> 172.21.15.6:0/71740740 conn(0x7eff640fb7c0 msgr2=0x7eff64103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:05:13.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.704+0000 7eff6c702700 1 -- 172.21.15.6:0/71740740 shutdown_connections 2024-03-24T12:05:13.705 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:13.704+0000 7eff6c702700 1 -- 172.21.15.6:0/71740740 wait complete. 2024-03-24T12:05:13.705 INFO:teuthology.orchestra.run.smithi006.stderr:adjusted tunables profile to default 2024-03-24T12:05:13.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:13 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/71740740' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-24T12:05:14.310 INFO:tasks.cephadm:Adding mon.a on smithi006 2024-03-24T12:05:14.310 INFO:tasks.cephadm:Adding mon.c on smithi006 2024-03-24T12:05:14.310 INFO:tasks.cephadm:Adding mon.b on smithi090 2024-03-24T12:05:14.310 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph orch apply mon '3;smithi006:172.21.15.6=a;smithi006:[v2:172.21.15.6:3301,v1:172.21.15.6:6790]=c;smithi090:172.21.15.90=b' 2024-03-24T12:05:14.547 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-24T12:05:14.593 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T12:05:14.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/71740740' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-24T12:05:14.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:14 smithi006 ceph-mon[30108]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T12:05:14.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:14 smithi006 ceph-mon[30108]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:14.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:14 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:14.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:14 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:14.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:14 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:14.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:14 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:14.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:14 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:05:14.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:14 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:14.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:14 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:05:15.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:15 smithi006 ceph-mon[30108]: Updating smithi090:/etc/ceph/ceph.conf 2024-03-24T12:05:15.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:15 smithi006 ceph-mon[30108]: Updating smithi090:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T12:05:15.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:15 smithi006 ceph-mon[30108]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-03-24T12:05:15.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:15 smithi006 ceph-mon[30108]: Updating smithi090:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T12:05:15.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:15 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:15.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:15 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:15.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:15 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:16.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.048+0000 7fa6cbed6700 1 -- 172.21.15.90:0/252451945 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6c4107650 msgr2=0x7fa6c4107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:16.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.048+0000 7fa6cbed6700 1 --2- 172.21.15.90:0/252451945 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6c4107650 0x7fa6c4107a30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000b0a0 tx=0x7fa6c000b3b0 comp rx=0 tx=0).stop 2024-03-24T12:05:16.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.048+0000 7fa6cbed6700 1 -- 172.21.15.90:0/252451945 shutdown_connections 2024-03-24T12:05:16.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.048+0000 7fa6cbed6700 1 --2- 172.21.15.90:0/252451945 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6c4107650 0x7fa6c4107a30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:16.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.048+0000 7fa6cbed6700 1 -- 172.21.15.90:0/252451945 >> 172.21.15.90:0/252451945 conn(0x7fa6c40fd180 msgr2=0x7fa6c40ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:05:16.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.048+0000 7fa6cbed6700 1 -- 172.21.15.90:0/252451945 shutdown_connections 2024-03-24T12:05:16.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.048+0000 7fa6cbed6700 1 -- 172.21.15.90:0/252451945 wait complete. 2024-03-24T12:05:16.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.049+0000 7fa6cbed6700 1 Processor -- start 2024-03-24T12:05:16.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.049+0000 7fa6cbed6700 1 -- start start 2024-03-24T12:05:16.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.049+0000 7fa6cbed6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6c4107650 0x7fa6c4196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:16.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.049+0000 7fa6cbed6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c41969d0 con 0x7fa6c4107650 2024-03-24T12:05:16.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.049+0000 7fa6c9c72700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6c4107650 0x7fa6c4196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:16.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.049+0000 7fa6c9c72700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6c4107650 0x7fa6c4196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.90:52030/0 (socket says 172.21.15.90:52030) 2024-03-24T12:05:16.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.049+0000 7fa6c9c72700 1 -- 172.21.15.90:0/1843051141 learned_addr learned my addr 172.21.15.90:0/1843051141 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-24T12:05:16.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.050+0000 7fa6c9c72700 1 -- 172.21.15.90:0/1843051141 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c0009d00 con 0x7fa6c4107650 2024-03-24T12:05:16.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.050+0000 7fa6c9c72700 1 --2- 172.21.15.90:0/1843051141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6c4107650 0x7fa6c4196490 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009610 tx=0x7fa6c000bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:05:16.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.050+0000 7fa6baffd700 1 -- 172.21.15.90:0/1843051141 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6c000e040 con 0x7fa6c4107650 2024-03-24T12:05:16.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.050+0000 7fa6cbed6700 1 -- 172.21.15.90:0/1843051141 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c4196bd0 con 0x7fa6c4107650 2024-03-24T12:05:16.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.050+0000 7fa6baffd700 1 -- 172.21.15.90:0/1843051141 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fa6c0003c10 con 0x7fa6c4107650 2024-03-24T12:05:16.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.050+0000 7fa6baffd700 1 -- 172.21.15.90:0/1843051141 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6c001d470 con 0x7fa6c4107650 2024-03-24T12:05:16.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.050+0000 7fa6cbed6700 1 -- 172.21.15.90:0/1843051141 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c4197050 con 0x7fa6c4107650 2024-03-24T12:05:16.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.051+0000 7fa6cbed6700 1 -- 172.21.15.90:0/1843051141 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c40632a0 con 0x7fa6c4107650 2024-03-24T12:05:16.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.051+0000 7fa6baffd700 1 -- 172.21.15.90:0/1843051141 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7fa6c0019070 con 0x7fa6c4107650 2024-03-24T12:05:16.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.051+0000 7fa6baffd700 1 --2- 172.21.15.90:0/1843051141 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fa6b0038d50 0x7fa6b003b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:16.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.051+0000 7fa6baffd700 1 -- 172.21.15.90:0/1843051141 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa6c004cdd0 con 0x7fa6c4107650 2024-03-24T12:05:16.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.051+0000 7fa6c9471700 1 --2- 172.21.15.90:0/1843051141 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fa6b0038d50 0x7fa6b003b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:16.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.052+0000 7fa6c9471700 1 --2- 172.21.15.90:0/1843051141 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fa6b0038d50 0x7fa6b003b210 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4006fd0 tx=0x7fa6b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:05:16.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.056+0000 7fa6baffd700 1 -- 172.21.15.90:0/1843051141 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa6c0004040 con 0x7fa6c4107650 2024-03-24T12:05:16.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.259+0000 7fa6cbed6700 1 -- 172.21.15.90:0/1843051141 --> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi006:172.21.15.6=a;smithi006:[v2:172.21.15.6:3301,v1:172.21.15.6:6790]=c;smithi090:172.21.15.90=b", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6c410af20 con 0x7fa6b0038d50 2024-03-24T12:05:16.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.265+0000 7fa6baffd700 1 -- 172.21.15.90:0/1843051141 <== mgr.14152 v2:172.21.15.6:6800/3847865179 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa6c410af20 con 0x7fa6b0038d50 2024-03-24T12:05:16.265 INFO:teuthology.orchestra.run.smithi090.stdout:Scheduled mon update... 2024-03-24T12:05:16.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.267+0000 7fa6cbed6700 1 -- 172.21.15.90:0/1843051141 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fa6b0038d50 msgr2=0x7fa6b003b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:16.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.267+0000 7fa6cbed6700 1 --2- 172.21.15.90:0/1843051141 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fa6b0038d50 0x7fa6b003b210 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4006fd0 tx=0x7fa6b4006e40 comp rx=0 tx=0).stop 2024-03-24T12:05:16.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.267+0000 7fa6cbed6700 1 -- 172.21.15.90:0/1843051141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6c4107650 msgr2=0x7fa6c4196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:16.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.267+0000 7fa6cbed6700 1 --2- 172.21.15.90:0/1843051141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6c4107650 0x7fa6c4196490 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009610 tx=0x7fa6c000bd40 comp rx=0 tx=0).stop 2024-03-24T12:05:16.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.267+0000 7fa6cbed6700 1 -- 172.21.15.90:0/1843051141 shutdown_connections 2024-03-24T12:05:16.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.267+0000 7fa6cbed6700 1 --2- 172.21.15.90:0/1843051141 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fa6b0038d50 0x7fa6b003b210 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:16.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.267+0000 7fa6cbed6700 1 --2- 172.21.15.90:0/1843051141 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6c4107650 0x7fa6c4196490 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:16.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.267+0000 7fa6cbed6700 1 -- 172.21.15.90:0/1843051141 >> 172.21.15.90:0/1843051141 conn(0x7fa6c40fd180 msgr2=0x7fa6c4109570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:05:16.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.267+0000 7fa6cbed6700 1 -- 172.21.15.90:0/1843051141 shutdown_connections 2024-03-24T12:05:16.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:16.267+0000 7fa6cbed6700 1 -- 172.21.15.90:0/1843051141 wait complete. 2024-03-24T12:05:16.822 DEBUG:teuthology.orchestra.run.smithi006:mon.c> sudo journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mon.c.service 2024-03-24T12:05:16.825 DEBUG:teuthology.orchestra.run.smithi090:mon.b> sudo journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mon.b.service 2024-03-24T12:05:16.828 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-24T12:05:16.828 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T12:05:17.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:16 smithi006 ceph-mon[30108]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:17.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:16 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:17.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:16 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:05:17.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:16 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:17.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:16 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:05:17.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:16 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:17.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:16 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T12:05:17.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:16 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:17.520 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T12:05:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:17 smithi006 ceph-mon[30108]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi006:172.21.15.6=a;smithi006:[v2:172.21.15.6:3301,v1:172.21.15.6:6790]=c;smithi090:172.21.15.90=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:05:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:17 smithi006 ceph-mon[30108]: Saving service mon spec with placement smithi006:172.21.15.6=a;smithi006:[v2:172.21.15.6:3301,v1:172.21.15.6:6790]=c;smithi090:172.21.15.90=b;count:3 2024-03-24T12:05:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:17 smithi006 ceph-mon[30108]: Deploying daemon mon.b on smithi090 2024-03-24T12:05:19.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:18 smithi006 ceph-mon[30108]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:20.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:19 smithi006 ceph-mon[30108]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:20.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.298+0000 7f9adcb4c700 1 -- 172.21.15.90:0/3465482258 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ad8107650 msgr2=0x7f9ad8107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:20.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.298+0000 7f9adcb4c700 1 --2- 172.21.15.90:0/3465482258 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ad8107650 0x7f9ad8107a30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009910 tx=0x7f9ac8009c20 comp rx=0 tx=0).stop 2024-03-24T12:05:20.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.298+0000 7f9adcb4c700 1 -- 172.21.15.90:0/3465482258 shutdown_connections 2024-03-24T12:05:20.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.298+0000 7f9adcb4c700 1 --2- 172.21.15.90:0/3465482258 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ad8107650 0x7f9ad8107a30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:20.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.298+0000 7f9adcb4c700 1 -- 172.21.15.90:0/3465482258 >> 172.21.15.90:0/3465482258 conn(0x7f9ad80fd160 msgr2=0x7f9ad80ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:05:20.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.298+0000 7f9adcb4c700 1 -- 172.21.15.90:0/3465482258 shutdown_connections 2024-03-24T12:05:20.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.299+0000 7f9adcb4c700 1 -- 172.21.15.90:0/3465482258 wait complete. 2024-03-24T12:05:20.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.299+0000 7f9adcb4c700 1 Processor -- start 2024-03-24T12:05:20.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.299+0000 7f9adcb4c700 1 -- start start 2024-03-24T12:05:20.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.300+0000 7f9adcb4c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ad8107650 0x7f9ad8196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:20.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.300+0000 7f9adcb4c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad8196a40 con 0x7f9ad8107650 2024-03-24T12:05:20.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.300+0000 7f9ad659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ad8107650 0x7f9ad8196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:20.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.300+0000 7f9ad659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ad8107650 0x7f9ad8196500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.90:52052/0 (socket says 172.21.15.90:52052) 2024-03-24T12:05:20.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.300+0000 7f9ad659c700 1 -- 172.21.15.90:0/1778168033 learned_addr learned my addr 172.21.15.90:0/1778168033 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-24T12:05:20.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.300+0000 7f9ad659c700 1 -- 172.21.15.90:0/1778168033 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac80095f0 con 0x7f9ad8107650 2024-03-24T12:05:20.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.300+0000 7f9ad659c700 1 --2- 172.21.15.90:0/1778168033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ad8107650 0x7f9ad8196500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8005660 tx=0x7f9ac8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:05:20.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.300+0000 7f9ac77fe700 1 -- 172.21.15.90:0/1778168033 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ac801d070 con 0x7f9ad8107650 2024-03-24T12:05:20.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.300+0000 7f9adcb4c700 1 -- 172.21.15.90:0/1778168033 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad8196ca0 con 0x7f9ad8107650 2024-03-24T12:05:20.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.300+0000 7f9ac77fe700 1 -- 172.21.15.90:0/1778168033 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f9ac800b6f0 con 0x7f9ad8107650 2024-03-24T12:05:20.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.301+0000 7f9ac77fe700 1 -- 172.21.15.90:0/1778168033 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ac80037f0 con 0x7f9ad8107650 2024-03-24T12:05:20.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.301+0000 7f9adcb4c700 1 -- 172.21.15.90:0/1778168033 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad8197120 con 0x7f9ad8107650 2024-03-24T12:05:20.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.301+0000 7f9ac77fe700 1 -- 172.21.15.90:0/1778168033 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7f9ac8003a60 con 0x7f9ad8107650 2024-03-24T12:05:20.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.302+0000 7f9adcb4c700 1 -- 172.21.15.90:0/1778168033 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad80632a0 con 0x7f9ad8107650 2024-03-24T12:05:20.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.302+0000 7f9ac77fe700 1 --2- 172.21.15.90:0/1778168033 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f9ac0038d50 0x7f9ac003b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:20.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.302+0000 7f9ac77fe700 1 -- 172.21.15.90:0/1778168033 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9ac804cb60 con 0x7f9ad8107650 2024-03-24T12:05:20.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.302+0000 7f9ad5d9b700 1 --2- 172.21.15.90:0/1778168033 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f9ac0038d50 0x7f9ac003b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:20.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.302+0000 7f9ad5d9b700 1 --2- 172.21.15.90:0/1778168033 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f9ac0038d50 0x7f9ac003b210 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9acc006fd0 tx=0x7f9acc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:05:20.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.307+0000 7f9ac77fe700 1 -- 172.21.15.90:0/1778168033 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9ac8029360 con 0x7f9ad8107650 2024-03-24T12:05:20.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.602+0000 7f9adcb4c700 1 -- 172.21.15.90:0/1778168033 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9ad8197860 con 0x7f9ad8107650 2024-03-24T12:05:20.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.603+0000 7f9ac77fe700 1 -- 172.21.15.90:0/1778168033 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+741 (secure 0 0 0) 0x7f9ac8026030 con 0x7f9ad8107650 2024-03-24T12:05:20.603 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:05:20.603 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":1,"fsid":"66d0ad74-e9d6-11ee-95cd-87774f69a715","modified":"2024-03-24T12:03:39.411497Z","created":"2024-03-24T12:03:39.411497Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-24T12:05:20.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.605+0000 7f9adcb4c700 1 -- 172.21.15.90:0/1778168033 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f9ac0038d50 msgr2=0x7f9ac003b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:20.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.605+0000 7f9adcb4c700 1 --2- 172.21.15.90:0/1778168033 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f9ac0038d50 0x7f9ac003b210 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9acc006fd0 tx=0x7f9acc006e40 comp rx=0 tx=0).stop 2024-03-24T12:05:20.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.605+0000 7f9adcb4c700 1 -- 172.21.15.90:0/1778168033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ad8107650 msgr2=0x7f9ad8196500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:20.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.605+0000 7f9adcb4c700 1 --2- 172.21.15.90:0/1778168033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ad8107650 0x7f9ad8196500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8005660 tx=0x7f9ac8005740 comp rx=0 tx=0).stop 2024-03-24T12:05:20.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.605+0000 7f9adcb4c700 1 -- 172.21.15.90:0/1778168033 shutdown_connections 2024-03-24T12:05:20.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.605+0000 7f9adcb4c700 1 --2- 172.21.15.90:0/1778168033 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f9ac0038d50 0x7f9ac003b210 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:20.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.605+0000 7f9adcb4c700 1 --2- 172.21.15.90:0/1778168033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ad8107650 0x7f9ad8196500 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:20.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.605+0000 7f9adcb4c700 1 -- 172.21.15.90:0/1778168033 >> 172.21.15.90:0/1778168033 conn(0x7f9ad80fd160 msgr2=0x7f9ad81094e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:05:20.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.605+0000 7f9adcb4c700 1 -- 172.21.15.90:0/1778168033 shutdown_connections 2024-03-24T12:05:20.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:20.605+0000 7f9adcb4c700 1 -- 172.21.15.90:0/1778168033 wait complete. 2024-03-24T12:05:20.606 INFO:teuthology.orchestra.run.smithi090.stderr:dumped monmap epoch 1 2024-03-24T12:05:20.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:20 smithi090 systemd[1]: Starting Ceph mon.b for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:05:21.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:20 smithi006 ceph-mon[30108]: from='client.? 172.21.15.90:0/1778168033' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T12:05:21.518 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 podman[33607]: 2024-03-24 12:05:21.34940931 +0000 UTC m=+0.100008804 container create f1cf4ff3f7c6c54498ea2f5bfe782512af3d8322b2140a50b062fae894a1bf6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-24T12:05:21.518 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 podman[33607]: 2024-03-24 12:05:21.267026816 +0000 UTC m=+0.017626318 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-24T12:05:21.518 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 podman[33607]: 2024-03-24 12:05:21.518615089 +0000 UTC m=+0.269214588 container init f1cf4ff3f7c6c54498ea2f5bfe782512af3d8322b2140a50b062fae894a1bf6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-24T12:05:21.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 podman[33607]: 2024-03-24 12:05:21.524518033 +0000 UTC m=+0.275117528 container start f1cf4ff3f7c6c54498ea2f5bfe782512af3d8322b2140a50b062fae894a1bf6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-b, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-24T12:05:21.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: set uid:gid to 167:167 (ceph:ceph) 2024-03-24T12:05:21.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev), process ceph-mon, pid 7 2024-03-24T12:05:21.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: pidfile_write: ignore empty --pid-file 2024-03-24T12:05:21.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: load: jerasure load: lrc 2024-03-24T12:05:21.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: RocksDB version: 7.9.2 2024-03-24T12:05:21.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Git sha 0 2024-03-24T12:05:21.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Compile date 2024-03-16 10:36:11 2024-03-24T12:05:21.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: DB SUMMARY 2024-03-24T12:05:21.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: DB Session ID: 7H0APW4OOOY21MMT0JYU 2024-03-24T12:05:21.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: CURRENT file: CURRENT 2024-03-24T12:05:21.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: IDENTITY file: IDENTITY 2024-03-24T12:05:21.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-24T12:05:21.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-03-24T12:05:21.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-03-24T12:05:21.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.error_if_exists: 0 2024-03-24T12:05:21.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.create_if_missing: 0 2024-03-24T12:05:21.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.paranoid_checks: 1 2024-03-24T12:05:21.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-24T12:05:21.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-24T12:05:21.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-24T12:05:21.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.env: 0x562239870ae0 2024-03-24T12:05:21.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.fs: PosixFileSystem 2024-03-24T12:05:21.931 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.info_log: 0x56223aef58c0 2024-03-24T12:05:21.931 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_file_opening_threads: 16 2024-03-24T12:05:21.931 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.statistics: (nil) 2024-03-24T12:05:21.931 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.use_fsync: 0 2024-03-24T12:05:21.931 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_log_file_size: 0 2024-03-24T12:05:21.931 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-24T12:05:21.931 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-24T12:05:21.931 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.keep_log_file_num: 1000 2024-03-24T12:05:21.931 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.recycle_log_file_num: 0 2024-03-24T12:05:21.931 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.allow_fallocate: 1 2024-03-24T12:05:21.933 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.allow_mmap_reads: 0 2024-03-24T12:05:21.933 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.allow_mmap_writes: 0 2024-03-24T12:05:21.933 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.use_direct_reads: 0 2024-03-24T12:05:21.933 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-24T12:05:21.933 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.create_missing_column_families: 0 2024-03-24T12:05:21.933 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.db_log_dir: 2024-03-24T12:05:21.934 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.wal_dir: 2024-03-24T12:05:21.934 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-24T12:05:21.934 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-24T12:05:21.934 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-24T12:05:21.934 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-24T12:05:21.934 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-24T12:05:21.934 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-24T12:05:21.934 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.advise_random_on_open: 1 2024-03-24T12:05:21.934 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.db_write_buffer_size: 0 2024-03-24T12:05:21.934 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.write_buffer_manager: 0x56223aec9400 2024-03-24T12:05:21.934 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-24T12:05:21.934 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-24T12:05:21.935 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-24T12:05:21.935 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.rate_limiter: (nil) 2024-03-24T12:05:21.935 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-24T12:05:21.935 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.wal_recovery_mode: 2 2024-03-24T12:05:21.935 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.enable_thread_tracking: 0 2024-03-24T12:05:21.935 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.enable_pipelined_write: 0 2024-03-24T12:05:21.935 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.unordered_write: 0 2024-03-24T12:05:21.935 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-24T12:05:21.935 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-24T12:05:21.935 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-24T12:05:21.935 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-24T12:05:21.935 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.row_cache: None 2024-03-24T12:05:21.936 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.wal_filter: None 2024-03-24T12:05:21.936 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-24T12:05:21.936 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.allow_ingest_behind: 0 2024-03-24T12:05:21.936 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.two_write_queues: 0 2024-03-24T12:05:21.936 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.manual_wal_flush: 0 2024-03-24T12:05:21.936 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.wal_compression: 0 2024-03-24T12:05:21.936 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.atomic_flush: 0 2024-03-24T12:05:21.936 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-24T12:05:21.936 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-24T12:05:21.936 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-24T12:05:21.936 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.log_readahead_size: 0 2024-03-24T12:05:21.936 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-24T12:05:21.937 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.best_efforts_recovery: 0 2024-03-24T12:05:21.937 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-24T12:05:21.937 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-24T12:05:21.937 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.allow_data_in_errors: 0 2024-03-24T12:05:21.937 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.db_host_id: __hostname__ 2024-03-24T12:05:21.937 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-24T12:05:21.937 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_background_jobs: 2 2024-03-24T12:05:21.937 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_background_compactions: -1 2024-03-24T12:05:21.937 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_subcompactions: 1 2024-03-24T12:05:21.937 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-24T12:05:21.937 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-24T12:05:21.937 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-24T12:05:21.938 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_total_wal_size: 0 2024-03-24T12:05:21.938 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-24T12:05:21.938 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-24T12:05:21.938 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-24T12:05:21.938 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-24T12:05:21.938 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_open_files: -1 2024-03-24T12:05:21.938 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.bytes_per_sync: 0 2024-03-24T12:05:21.938 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-24T12:05:21.938 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-24T12:05:21.938 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.compaction_readahead_size: 0 2024-03-24T12:05:21.938 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_background_flushes: -1 2024-03-24T12:05:21.938 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Compression algorithms supported: 2024-03-24T12:05:21.938 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-24T12:05:21.939 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: kZSTD supported: 0 2024-03-24T12:05:21.939 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: kXpressCompression supported: 0 2024-03-24T12:05:21.939 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: kLZ4HCCompression supported: 1 2024-03-24T12:05:21.939 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: kZlibCompression supported: 1 2024-03-24T12:05:21.939 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: kSnappyCompression supported: 1 2024-03-24T12:05:21.939 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: kLZ4Compression supported: 1 2024-03-24T12:05:21.939 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: kBZip2Compression supported: 0 2024-03-24T12:05:21.939 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-24T12:05:21.939 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-24T12:05:21.939 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-03-24T12:05:21.939 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-24T12:05:21.940 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-24T12:05:21.940 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.merge_operator: 2024-03-24T12:05:21.940 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.compaction_filter: None 2024-03-24T12:05:21.940 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.compaction_filter_factory: None 2024-03-24T12:05:21.940 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.sst_partitioner_factory: None 2024-03-24T12:05:21.940 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-24T12:05:21.940 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-24T12:05:21.940 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56223aef59c0) 2024-03-24T12:05:21.940 INFO:journalctl@ceph.mon.b.smithi090.stdout: cache_index_and_filter_blocks: 1 2024-03-24T12:05:21.940 INFO:journalctl@ceph.mon.b.smithi090.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-24T12:05:21.940 INFO:journalctl@ceph.mon.b.smithi090.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-24T12:05:21.940 INFO:journalctl@ceph.mon.b.smithi090.stdout: pin_top_level_index_and_filter: 1 2024-03-24T12:05:21.941 INFO:journalctl@ceph.mon.b.smithi090.stdout: index_type: 0 2024-03-24T12:05:21.941 INFO:journalctl@ceph.mon.b.smithi090.stdout: data_block_index_type: 0 2024-03-24T12:05:21.941 INFO:journalctl@ceph.mon.b.smithi090.stdout: index_shortening: 1 2024-03-24T12:05:21.941 INFO:journalctl@ceph.mon.b.smithi090.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-24T12:05:21.941 INFO:journalctl@ceph.mon.b.smithi090.stdout: checksum: 4 2024-03-24T12:05:21.941 INFO:journalctl@ceph.mon.b.smithi090.stdout: no_block_cache: 0 2024-03-24T12:05:21.941 INFO:journalctl@ceph.mon.b.smithi090.stdout: block_cache: 0x56223aef9090 2024-03-24T12:05:21.941 INFO:journalctl@ceph.mon.b.smithi090.stdout: block_cache_name: BinnedLRUCache 2024-03-24T12:05:21.941 INFO:journalctl@ceph.mon.b.smithi090.stdout: block_cache_options: 2024-03-24T12:05:21.941 INFO:journalctl@ceph.mon.b.smithi090.stdout: capacity : 536870912 2024-03-24T12:05:21.941 INFO:journalctl@ceph.mon.b.smithi090.stdout: num_shard_bits : 4 2024-03-24T12:05:21.941 INFO:journalctl@ceph.mon.b.smithi090.stdout: strict_capacity_limit : 0 2024-03-24T12:05:21.941 INFO:journalctl@ceph.mon.b.smithi090.stdout: high_pri_pool_ratio: 0.000 2024-03-24T12:05:21.942 INFO:journalctl@ceph.mon.b.smithi090.stdout: block_cache_compressed: (nil) 2024-03-24T12:05:21.942 INFO:journalctl@ceph.mon.b.smithi090.stdout: persistent_cache: (nil) 2024-03-24T12:05:21.942 INFO:journalctl@ceph.mon.b.smithi090.stdout: block_size: 4096 2024-03-24T12:05:21.942 INFO:journalctl@ceph.mon.b.smithi090.stdout: block_size_deviation: 10 2024-03-24T12:05:21.942 INFO:journalctl@ceph.mon.b.smithi090.stdout: block_restart_interval: 16 2024-03-24T12:05:21.942 INFO:journalctl@ceph.mon.b.smithi090.stdout: index_block_restart_interval: 1 2024-03-24T12:05:21.942 INFO:journalctl@ceph.mon.b.smithi090.stdout: metadata_block_size: 4096 2024-03-24T12:05:21.942 INFO:journalctl@ceph.mon.b.smithi090.stdout: partition_filters: 0 2024-03-24T12:05:21.942 INFO:journalctl@ceph.mon.b.smithi090.stdout: use_delta_encoding: 1 2024-03-24T12:05:21.943 INFO:journalctl@ceph.mon.b.smithi090.stdout: filter_policy: bloomfilter 2024-03-24T12:05:21.943 INFO:journalctl@ceph.mon.b.smithi090.stdout: whole_key_filtering: 1 2024-03-24T12:05:21.943 INFO:journalctl@ceph.mon.b.smithi090.stdout: verify_compression: 0 2024-03-24T12:05:21.943 INFO:journalctl@ceph.mon.b.smithi090.stdout: read_amp_bytes_per_bit: 0 2024-03-24T12:05:21.943 INFO:journalctl@ceph.mon.b.smithi090.stdout: format_version: 5 2024-03-24T12:05:21.943 INFO:journalctl@ceph.mon.b.smithi090.stdout: enable_index_compression: 1 2024-03-24T12:05:21.943 INFO:journalctl@ceph.mon.b.smithi090.stdout: block_align: 0 2024-03-24T12:05:21.943 INFO:journalctl@ceph.mon.b.smithi090.stdout: max_auto_readahead_size: 262144 2024-03-24T12:05:21.943 INFO:journalctl@ceph.mon.b.smithi090.stdout: prepopulate_block_cache: 0 2024-03-24T12:05:21.943 INFO:journalctl@ceph.mon.b.smithi090.stdout: initial_auto_readahead_size: 8192 2024-03-24T12:05:21.943 INFO:journalctl@ceph.mon.b.smithi090.stdout: num_file_reads_for_auto_readahead: 2 2024-03-24T12:05:21.944 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.write_buffer_size: 33554432 2024-03-24T12:05:21.944 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_write_buffer_number: 2 2024-03-24T12:05:21.944 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.compression: NoCompression 2024-03-24T12:05:21.944 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.bottommost_compression: Disabled 2024-03-24T12:05:21.944 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.prefix_extractor: nullptr 2024-03-24T12:05:21.944 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-24T12:05:21.944 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.num_levels: 7 2024-03-24T12:05:21.944 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-24T12:05:21.944 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-24T12:05:21.944 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-24T12:05:21.944 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-24T12:05:21.944 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-24T12:05:21.944 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-24T12:05:21.945 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-24T12:05:21.945 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-24T12:05:21.945 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-24T12:05:21.945 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-24T12:05:21.945 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-24T12:05:21.945 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-24T12:05:21.945 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-24T12:05:21.945 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.compression_opts.level: 32767 2024-03-24T12:05:21.945 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.compression_opts.strategy: 0 2024-03-24T12:05:21.945 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-24T12:05:21.945 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-24T12:05:21.946 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-24T12:05:21.946 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-24T12:05:21.946 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.compression_opts.enabled: false 2024-03-24T12:05:21.946 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-24T12:05:21.946 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-24T12:05:21.946 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-24T12:05:21.946 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-24T12:05:21.946 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.target_file_size_base: 67108864 2024-03-24T12:05:21.946 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-24T12:05:21.946 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-24T12:05:21.946 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-24T12:05:21.946 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-24T12:05:21.947 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-24T12:05:21.947 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-24T12:05:21.947 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-24T12:05:21.947 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-24T12:05:21.947 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-24T12:05:21.947 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-24T12:05:21.947 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-24T12:05:21.947 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-24T12:05:21.947 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-24T12:05:21.947 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-24T12:05:21.947 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.arena_block_size: 1048576 2024-03-24T12:05:21.947 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-24T12:05:21.948 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-24T12:05:21.948 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.disable_auto_compactions: 0 2024-03-24T12:05:21.948 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-24T12:05:21.948 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-24T12:05:21.948 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-24T12:05:21.948 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-24T12:05:21.948 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-24T12:05:21.948 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-24T12:05:21.948 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-24T12:05:21.948 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-24T12:05:21.948 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-24T12:05:21.948 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-24T12:05:21.949 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.table_properties_collectors: 2024-03-24T12:05:21.949 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.inplace_update_support: 0 2024-03-24T12:05:21.949 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-24T12:05:21.949 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-24T12:05:21.949 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-24T12:05:21.949 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-24T12:05:21.949 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.bloom_locality: 0 2024-03-24T12:05:21.949 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.max_successive_merges: 0 2024-03-24T12:05:21.949 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-24T12:05:21.949 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.paranoid_file_checks: 0 2024-03-24T12:05:21.949 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.force_consistency_checks: 1 2024-03-24T12:05:21.949 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.report_bg_io_stats: 0 2024-03-24T12:05:21.950 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.ttl: 2592000 2024-03-24T12:05:21.950 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-24T12:05:21.950 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-24T12:05:21.950 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-24T12:05:21.950 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.enable_blob_files: false 2024-03-24T12:05:21.950 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.min_blob_size: 0 2024-03-24T12:05:21.950 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.blob_file_size: 268435456 2024-03-24T12:05:21.950 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-24T12:05:21.950 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-24T12:05:21.950 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-24T12:05:21.950 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-24T12:05:21.951 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-24T12:05:21.951 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.blob_file_starting_level: 0 2024-03-24T12:05:21.951 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-24T12:05:21.951 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-24T12:05:21.951 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-24T12:05:21.951 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: de3d536b-574f-4ad5-b694-ea1fd94d7e6f 2024-03-24T12:05:21.951 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711281921554250, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-24T12:05:21.951 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-24T12:05:21.951 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711281921554656, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711281921, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "de3d536b-574f-4ad5-b694-ea1fd94d7e6f", "db_session_id": "7H0APW4OOOY21MMT0JYU", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-24T12:05:21.951 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711281921554733, "job": 1, "event": "recovery_finished"} 2024-03-24T12:05:21.951 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-24T12:05:21.951 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-24T12:05:21.952 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56223afea000 2024-03-24T12:05:21.952 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: DB pointer 0x56223afd4000 2024-03-24T12:05:21.952 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-24T12:05:21.952 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-24T12:05:21.952 INFO:journalctl@ceph.mon.b.smithi090.stdout: ** DB Stats ** 2024-03-24T12:05:21.952 INFO:journalctl@ceph.mon.b.smithi090.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T12:05:21.952 INFO:journalctl@ceph.mon.b.smithi090.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-24T12:05:21.952 INFO:journalctl@ceph.mon.b.smithi090.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T12:05:21.952 INFO:journalctl@ceph.mon.b.smithi090.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T12:05:21.952 INFO:journalctl@ceph.mon.b.smithi090.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-24T12:05:21.952 INFO:journalctl@ceph.mon.b.smithi090.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-24T12:05:21.952 INFO:journalctl@ceph.mon.b.smithi090.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-24T12:05:21.953 INFO:journalctl@ceph.mon.b.smithi090.stdout: 2024-03-24T12:05:21.953 INFO:journalctl@ceph.mon.b.smithi090.stdout: ** Compaction Stats [default] ** 2024-03-24T12:05:21.953 INFO:journalctl@ceph.mon.b.smithi090.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T12:05:21.953 INFO:journalctl@ceph.mon.b.smithi090.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-24T12:05:21.953 INFO:journalctl@ceph.mon.b.smithi090.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-24T12:05:21.953 INFO:journalctl@ceph.mon.b.smithi090.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-24T12:05:21.953 INFO:journalctl@ceph.mon.b.smithi090.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-24T12:05:21.953 INFO:journalctl@ceph.mon.b.smithi090.stdout: 2024-03-24T12:05:21.953 INFO:journalctl@ceph.mon.b.smithi090.stdout: ** Compaction Stats [default] ** 2024-03-24T12:05:21.953 INFO:journalctl@ceph.mon.b.smithi090.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-24T12:05:21.953 INFO:journalctl@ceph.mon.b.smithi090.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-24T12:05:21.954 INFO:journalctl@ceph.mon.b.smithi090.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-24T12:05:21.954 INFO:journalctl@ceph.mon.b.smithi090.stdout: 2024-03-24T12:05:21.954 INFO:journalctl@ceph.mon.b.smithi090.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-24T12:05:21.954 INFO:journalctl@ceph.mon.b.smithi090.stdout: 2024-03-24T12:05:21.954 INFO:journalctl@ceph.mon.b.smithi090.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-24T12:05:21.954 INFO:journalctl@ceph.mon.b.smithi090.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-24T12:05:21.954 INFO:journalctl@ceph.mon.b.smithi090.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-24T12:05:21.954 INFO:journalctl@ceph.mon.b.smithi090.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-24T12:05:21.954 INFO:journalctl@ceph.mon.b.smithi090.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-24T12:05:21.954 INFO:journalctl@ceph.mon.b.smithi090.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-24T12:05:21.954 INFO:journalctl@ceph.mon.b.smithi090.stdout: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T12:05:21.954 INFO:journalctl@ceph.mon.b.smithi090.stdout: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-24T12:05:21.955 INFO:journalctl@ceph.mon.b.smithi090.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-24T12:05:21.955 INFO:journalctl@ceph.mon.b.smithi090.stdout: Block cache BinnedLRUCache@0x56223aef9090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-24T12:05:21.955 INFO:journalctl@ceph.mon.b.smithi090.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-24T12:05:21.955 INFO:journalctl@ceph.mon.b.smithi090.stdout: 2024-03-24T12:05:21.955 INFO:journalctl@ceph.mon.b.smithi090.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-24T12:05:21.955 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-03-24T12:05:21.955 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: using public_addr v2:172.21.15.90:0/0 -> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] 2024-03-24T12:05:21.955 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: starting mon.b rank -1 at public addrs [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] at bind addrs [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 2024-03-24T12:05:21.955 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: mon.b@-1(???) e0 preinit fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 2024-03-24T12:05:21.955 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: mon.b@-1(synchronizing).mds e1 new map 2024-03-24T12:05:21.955 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-24T12:05:21.955 INFO:journalctl@ceph.mon.b.smithi090.stdout: e1 2024-03-24T12:05:21.955 INFO:journalctl@ceph.mon.b.smithi090.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-24T12:05:21.956 INFO:journalctl@ceph.mon.b.smithi090.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-24T12:05:21.956 INFO:journalctl@ceph.mon.b.smithi090.stdout: legacy client fscid: -1 2024-03-24T12:05:21.956 INFO:journalctl@ceph.mon.b.smithi090.stdout: 2024-03-24T12:05:21.956 INFO:journalctl@ceph.mon.b.smithi090.stdout: No filesystems configured 2024-03-24T12:05:21.956 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-24T12:05:21.956 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-24T12:05:21.956 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-24T12:05:21.956 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-24T12:05:21.956 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-24T12:05:21.956 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-24T12:05:21.956 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-24T12:05:21.956 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T12:05:21.956 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T12:05:21.957 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-24T12:05:21.957 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:21.957 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T12:05:21.957 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:21.957 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/71740740' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-24T12:05:21.957 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/71740740' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-24T12:05:21.957 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T12:05:21.957 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:21.957 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:21.957 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:21.958 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:21.958 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:21.958 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:05:21.958 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:21.958 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:05:21.958 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: Updating smithi090:/etc/ceph/ceph.conf 2024-03-24T12:05:21.958 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: Updating smithi090:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T12:05:21.958 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-03-24T12:05:21.958 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: Updating smithi090:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T12:05:21.958 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:21.958 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:21.959 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:21.959 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:21.959 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:21.959 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:05:21.959 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:21.959 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:05:21.959 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:21.959 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T12:05:21.959 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:21.959 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi006:172.21.15.6=a;smithi006:[v2:172.21.15.6:3301,v1:172.21.15.6:6790]=c;smithi090:172.21.15.90=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:05:21.959 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: Saving service mon spec with placement smithi006:172.21.15.6=a;smithi006:[v2:172.21.15.6:3301,v1:172.21.15.6:6790]=c;smithi090:172.21.15.90=b;count:3 2024-03-24T12:05:21.959 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: Deploying daemon mon.b on smithi090 2024-03-24T12:05:21.960 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:21.960 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:21.960 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.90:0/1778168033' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T12:05:21.960 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 ceph-mon[33646]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-24T12:05:21.960 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 bash[33607]: f1cf4ff3f7c6c54498ea2f5bfe782512af3d8322b2140a50b062fae894a1bf6d 2024-03-24T12:05:21.960 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:21 smithi090 systemd[1]: Started Ceph mon.b for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:05:22.224 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-24T12:05:22.225 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T12:05:22.458 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T12:05:26.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:26 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T12:05:26.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:26 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:05:26.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:26 smithi006 ceph-mon[30108]: mon.a calling monitor election 2024-03-24T12:05:26.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:26 smithi006 ceph-mon[30108]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:26.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:26 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:05:26.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:26 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:05:26.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:26 smithi006 ceph-mon[30108]: mon.b calling monitor election 2024-03-24T12:05:26.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:26 smithi006 ceph-mon[30108]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:26.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:26 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:05:26.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:26 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:05:26.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:26 smithi006 ceph-mon[30108]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:26.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:26 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:05:26.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:26 smithi006 ceph-mon[30108]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-03-24T12:05:26.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:26 smithi006 ceph-mon[30108]: monmap e2: 2 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],b=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T12:05:26.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:26 smithi006 ceph-mon[30108]: fsmap 2024-03-24T12:05:26.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:26 smithi006 ceph-mon[30108]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T12:05:26.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:26 smithi006 ceph-mon[30108]: mgrmap e13: y(active, since 40s) 2024-03-24T12:05:26.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:26 smithi006 ceph-mon[30108]: overall HEALTH_OK 2024-03-24T12:05:26.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:26 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:26.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:26 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:26.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:26 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:26.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:26 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T12:05:26.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:26 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:27.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:26 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T12:05:27.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:26 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:05:27.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:26 smithi090 ceph-mon[33646]: mon.a calling monitor election 2024-03-24T12:05:27.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:26 smithi090 ceph-mon[33646]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:27.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:26 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:05:27.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:26 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:05:27.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:26 smithi090 ceph-mon[33646]: mon.b calling monitor election 2024-03-24T12:05:27.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:26 smithi090 ceph-mon[33646]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:27.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:26 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:05:27.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:26 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:05:27.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:26 smithi090 ceph-mon[33646]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:27.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:26 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:05:27.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:26 smithi090 ceph-mon[33646]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-03-24T12:05:27.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:26 smithi090 ceph-mon[33646]: monmap e2: 2 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],b=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T12:05:27.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:26 smithi090 ceph-mon[33646]: fsmap 2024-03-24T12:05:27.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:26 smithi090 ceph-mon[33646]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T12:05:27.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:26 smithi090 ceph-mon[33646]: mgrmap e13: y(active, since 40s) 2024-03-24T12:05:27.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:26 smithi090 ceph-mon[33646]: overall HEALTH_OK 2024-03-24T12:05:27.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:26 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:27.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:26 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:27.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:26 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:27.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:26 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T12:05:27.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:26 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:27.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.252+0000 7f59341b5700 1 -- 172.21.15.90:0/4067126899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59180035c0 msgr2=0x7f5918005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:27.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.252+0000 7f59341b5700 1 --2- 172.21.15.90:0/4067126899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59180035c0 0x7f5918005a50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f592c069f20 tx=0x7f591c00feb0 comp rx=0 tx=0).stop 2024-03-24T12:05:27.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.252+0000 7f59341b5700 1 -- 172.21.15.90:0/4067126899 shutdown_connections 2024-03-24T12:05:27.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.252+0000 7f59341b5700 1 --2- 172.21.15.90:0/4067126899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59180035c0 0x7f5918005a50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:27.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.252+0000 7f59341b5700 1 -- 172.21.15.90:0/4067126899 >> 172.21.15.90:0/4067126899 conn(0x7f592c0fc180 msgr2=0x7f592c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:05:27.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.252+0000 7f59341b5700 1 -- 172.21.15.90:0/4067126899 shutdown_connections 2024-03-24T12:05:27.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.252+0000 7f59341b5700 1 -- 172.21.15.90:0/4067126899 wait complete. 2024-03-24T12:05:27.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.253+0000 7f59341b5700 1 Processor -- start 2024-03-24T12:05:27.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.253+0000 7f59341b5700 1 -- start start 2024-03-24T12:05:27.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.254+0000 7f59341b5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59180035c0 0x7f592c196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:27.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.254+0000 7f59341b5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f592c196bd0 0x7f592c19af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:27.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.254+0000 7f59341b5700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c197160 con 0x7f592c196bd0 2024-03-24T12:05:27.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.254+0000 7f59341b5700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c1972d0 con 0x7f59180035c0 2024-03-24T12:05:27.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.254+0000 7f5931f51700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59180035c0 0x7f592c196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:27.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.254+0000 7f5931f51700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59180035c0 0x7f592c196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54546/0 (socket says 172.21.15.90:54546) 2024-03-24T12:05:27.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.254+0000 7f5931f51700 1 -- 172.21.15.90:0/1942527973 learned_addr learned my addr 172.21.15.90:0/1942527973 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-24T12:05:27.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.254+0000 7f5931750700 1 --2- 172.21.15.90:0/1942527973 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f592c196bd0 0x7f592c19af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:27.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.254+0000 7f5931f51700 1 -- 172.21.15.90:0/1942527973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59180035c0 msgr2=0x7f592c196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-24T12:05:27.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.254+0000 7f5931f51700 1 -- 172.21.15.90:0/1942527973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59180035c0 msgr2=0x7f592c196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-24T12:05:27.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.254+0000 7f5931f51700 1 --2- 172.21.15.90:0/1942527973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59180035c0 0x7f592c196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-24T12:05:27.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.254+0000 7f5931f51700 1 --2- 172.21.15.90:0/1942527973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59180035c0 0x7f592c196690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T12:05:27.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.254+0000 7f5931750700 1 -- 172.21.15.90:0/1942527973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59180035c0 msgr2=0x7f592c196690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T12:05:27.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.254+0000 7f5931750700 1 --2- 172.21.15.90:0/1942527973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59180035c0 0x7f592c196690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:27.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.254+0000 7f5931750700 1 -- 172.21.15.90:0/1942527973 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c00f900 con 0x7f592c196bd0 2024-03-24T12:05:27.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.254+0000 7f5931750700 1 --2- 172.21.15.90:0/1942527973 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f592c196bd0 0x7f592c19af60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f592800b4d0 tx=0x7f592800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:05:27.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.255+0000 7f5922ffd700 1 -- 172.21.15.90:0/1942527973 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f5928011630 con 0x7f592c196bd0 2024-03-24T12:05:27.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.255+0000 7f5922ffd700 1 -- 172.21.15.90:0/1942527973 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f5928011c10 con 0x7f592c196bd0 2024-03-24T12:05:27.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.255+0000 7f59341b5700 1 -- 172.21.15.90:0/1942527973 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c19b500 con 0x7f592c196bd0 2024-03-24T12:05:27.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.255+0000 7f5922ffd700 1 -- 172.21.15.90:0/1942527973 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f5928010b80 con 0x7f592c196bd0 2024-03-24T12:05:27.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.255+0000 7f59341b5700 1 -- 172.21.15.90:0/1942527973 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c19ba50 con 0x7f592c196bd0 2024-03-24T12:05:27.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.255+0000 7f5922ffd700 1 -- 172.21.15.90:0/1942527973 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7f592801a400 con 0x7f592c196bd0 2024-03-24T12:05:27.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.256+0000 7f5922ffd700 1 --2- 172.21.15.90:0/1942527973 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f5910038af0 0x7f591003afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:27.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.256+0000 7f5922ffd700 1 -- 172.21.15.90:0/1942527973 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f592804c2e0 con 0x7f592c196bd0 2024-03-24T12:05:27.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.256+0000 7f5931f51700 1 --2- 172.21.15.90:0/1942527973 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f5910038af0 0x7f591003afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:27.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.256+0000 7f59341b5700 1 -- 172.21.15.90:0/1942527973 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592c190300 con 0x7f592c196bd0 2024-03-24T12:05:27.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.257+0000 7f5931f51700 1 --2- 172.21.15.90:0/1942527973 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f5910038af0 0x7f591003afb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f591c005850 tx=0x7f591c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:05:27.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.261+0000 7f5922ffd700 1 -- 172.21.15.90:0/1942527973 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5928010e40 con 0x7f592c196bd0 2024-03-24T12:05:27.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.534+0000 7f59341b5700 1 -- 172.21.15.90:0/1942527973 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f592c0632a0 con 0x7f592c196bd0 2024-03-24T12:05:27.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.535+0000 7f5922ffd700 1 -- 172.21.15.90:0/1942527973 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1007 (secure 0 0 0) 0x7f59280172c0 con 0x7f592c196bd0 2024-03-24T12:05:27.535 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:05:27.535 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":2,"fsid":"66d0ad74-e9d6-11ee-95cd-87774f69a715","modified":"2024-03-24T12:05:21.582088Z","created":"2024-03-24T12:03:39.411497Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-24T12:05:27.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.537+0000 7f59341b5700 1 -- 172.21.15.90:0/1942527973 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f5910038af0 msgr2=0x7f591003afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:27.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.537+0000 7f59341b5700 1 --2- 172.21.15.90:0/1942527973 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f5910038af0 0x7f591003afb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f591c005850 tx=0x7f591c005960 comp rx=0 tx=0).stop 2024-03-24T12:05:27.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.537+0000 7f59341b5700 1 -- 172.21.15.90:0/1942527973 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f592c196bd0 msgr2=0x7f592c19af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:27.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.537+0000 7f59341b5700 1 --2- 172.21.15.90:0/1942527973 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f592c196bd0 0x7f592c19af60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f592800b4d0 tx=0x7f592800b890 comp rx=0 tx=0).stop 2024-03-24T12:05:27.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.537+0000 7f59341b5700 1 -- 172.21.15.90:0/1942527973 shutdown_connections 2024-03-24T12:05:27.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.537+0000 7f59341b5700 1 --2- 172.21.15.90:0/1942527973 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f5910038af0 0x7f591003afb0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:27.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.537+0000 7f59341b5700 1 --2- 172.21.15.90:0/1942527973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59180035c0 0x7f592c196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:27.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.537+0000 7f59341b5700 1 --2- 172.21.15.90:0/1942527973 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f592c196bd0 0x7f592c19af60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:27.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.537+0000 7f59341b5700 1 -- 172.21.15.90:0/1942527973 >> 172.21.15.90:0/1942527973 conn(0x7f592c0fc180 msgr2=0x7f592c0fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:05:27.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.537+0000 7f59341b5700 1 -- 172.21.15.90:0/1942527973 shutdown_connections 2024-03-24T12:05:27.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:27.537+0000 7f59341b5700 1 -- 172.21.15.90:0/1942527973 wait complete. 2024-03-24T12:05:27.538 INFO:teuthology.orchestra.run.smithi090.stderr:dumped monmap epoch 2 2024-03-24T12:05:27.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:27 smithi090 ceph-mon[33646]: Deploying daemon mon.c on smithi006 2024-03-24T12:05:27.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:27 smithi090 ceph-mon[33646]: from='client.? 172.21.15.90:0/1942527973' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T12:05:27.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:27 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:05:27.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:27 smithi006 ceph-mon[30108]: Deploying daemon mon.c on smithi006 2024-03-24T12:05:27.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:27 smithi006 ceph-mon[30108]: from='client.? 172.21.15.90:0/1942527973' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T12:05:27.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:27 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:05:28.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:28 smithi006 ceph-mon[30108]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:29.052 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-24T12:05:29.053 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-24T12:05:29.083 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:28 smithi090 ceph-mon[33646]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:29.297 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T12:05:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:29 smithi006 ceph-mon[30108]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:30.158 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:29 smithi090 ceph-mon[33646]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:30.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:30 smithi006 systemd[1]: Started Ceph mon.c for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:05:35.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.270+0000 7fd0cb536700 1 -- 172.21.15.90:0/3225626802 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0b00035c0 msgr2=0x7fd0b0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:35.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.270+0000 7fd0cb536700 1 --2- 172.21.15.90:0/3225626802 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0b00035c0 0x7fd0b0005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4069f20 tx=0x7fd0c000fe60 comp rx=0 tx=0).stop 2024-03-24T12:05:35.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.270+0000 7fd0cb536700 1 -- 172.21.15.90:0/3225626802 shutdown_connections 2024-03-24T12:05:35.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.270+0000 7fd0cb536700 1 --2- 172.21.15.90:0/3225626802 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0b00035c0 0x7fd0b0005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:35.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.270+0000 7fd0cb536700 1 -- 172.21.15.90:0/3225626802 >> 172.21.15.90:0/3225626802 conn(0x7fd0c40fc180 msgr2=0x7fd0c40fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:05:35.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.270+0000 7fd0cb536700 1 -- 172.21.15.90:0/3225626802 shutdown_connections 2024-03-24T12:05:35.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.270+0000 7fd0cb536700 1 -- 172.21.15.90:0/3225626802 wait complete. 2024-03-24T12:05:35.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.271+0000 7fd0cb536700 1 Processor -- start 2024-03-24T12:05:35.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.271+0000 7fd0cb536700 1 -- start start 2024-03-24T12:05:35.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0cb536700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0b00035c0 0x7fd0c419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:35.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0cb536700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0c419d290 0x7fd0c41a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:35.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0cb536700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0c41a1b60 0x7fd0c41a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:35.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0cb536700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c419d980 con 0x7fd0c419d290 2024-03-24T12:05:35.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0cb536700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c41a98d0 con 0x7fd0b00035c0 2024-03-24T12:05:35.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0cb536700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c41a9a10 con 0x7fd0c41a1b60 2024-03-24T12:05:35.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0c92d2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0b00035c0 0x7fd0c419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:35.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0c8ad1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0c419d290 0x7fd0c41a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:35.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0c9ad3700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0c41a1b60 0x7fd0c41a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:35.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0c92d2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0b00035c0 0x7fd0c419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49008/0 (socket says 172.21.15.90:49008) 2024-03-24T12:05:35.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0c92d2700 1 -- 172.21.15.90:0/4163714777 learned_addr learned my addr 172.21.15.90:0/4163714777 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-24T12:05:35.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0c92d2700 1 -- 172.21.15.90:0/4163714777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0b00035c0 msgr2=0x7fd0c419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-24T12:05:35.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0c92d2700 1 -- 172.21.15.90:0/4163714777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0b00035c0 msgr2=0x7fd0c419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-24T12:05:35.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0c92d2700 1 --2- 172.21.15.90:0/4163714777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0b00035c0 0x7fd0c419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-24T12:05:35.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0c9ad3700 1 -- 172.21.15.90:0/4163714777 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0c41a1b60 msgr2=0x7fd0c41a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-03-24T12:05:35.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0c9ad3700 1 -- 172.21.15.90:0/4163714777 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0c41a1b60 msgr2=0x7fd0c41a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-24T12:05:35.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0c9ad3700 1 --2- 172.21.15.90:0/4163714777 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0c41a1b60 0x7fd0c41a2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-24T12:05:35.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0c92d2700 1 --2- 172.21.15.90:0/4163714777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0b00035c0 0x7fd0c419cd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T12:05:35.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0c9ad3700 1 --2- 172.21.15.90:0/4163714777 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0c41a1b60 0x7fd0c41a2010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T12:05:35.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0c8ad1700 1 -- 172.21.15.90:0/4163714777 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0c41a1b60 msgr2=0x7fd0c41a2010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T12:05:35.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0c8ad1700 1 --2- 172.21.15.90:0/4163714777 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0c41a1b60 0x7fd0c41a2010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:35.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0c8ad1700 1 -- 172.21.15.90:0/4163714777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0b00035c0 msgr2=0x7fd0c419cd50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T12:05:35.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0c8ad1700 1 --2- 172.21.15.90:0/4163714777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0b00035c0 0x7fd0c419cd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:35.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.272+0000 7fd0c8ad1700 1 -- 172.21.15.90:0/4163714777 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c000f8b0 con 0x7fd0c419d290 2024-03-24T12:05:35.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.273+0000 7fd0c8ad1700 1 --2- 172.21.15.90:0/4163714777 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0c419d290 0x7fd0c41a1620 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400b550 tx=0x7fd0b400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:05:35.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.275+0000 7fd0ba7fc700 1 -- 172.21.15.90:0/4163714777 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4004230 con 0x7fd0c419d290 2024-03-24T12:05:35.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.275+0000 7fd0ba7fc700 1 -- 172.21.15.90:0/4163714777 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd0b4004390 con 0x7fd0c419d290 2024-03-24T12:05:35.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.275+0000 7fd0cb536700 1 -- 172.21.15.90:0/4163714777 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c41a9cf0 con 0x7fd0c419d290 2024-03-24T12:05:35.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.275+0000 7fd0ba7fc700 1 -- 172.21.15.90:0/4163714777 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b40119d0 con 0x7fd0c419d290 2024-03-24T12:05:35.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.275+0000 7fd0cb536700 1 -- 172.21.15.90:0/4163714777 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c41aa1e0 con 0x7fd0c419d290 2024-03-24T12:05:35.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.276+0000 7fd0ba7fc700 1 -- 172.21.15.90:0/4163714777 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7fd0b4011b30 con 0x7fd0c419d290 2024-03-24T12:05:35.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.277+0000 7fd0cb536700 1 -- 172.21.15.90:0/4163714777 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c406a720 con 0x7fd0c419d290 2024-03-24T12:05:35.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.277+0000 7fd0ba7fc700 1 --2- 172.21.15.90:0/4163714777 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fd0a80391e0 0x7fd0a803b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:35.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.277+0000 7fd0ba7fc700 1 -- 172.21.15.90:0/4163714777 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd0b404b390 con 0x7fd0c419d290 2024-03-24T12:05:35.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.277+0000 7fd0c92d2700 1 --2- 172.21.15.90:0/4163714777 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fd0a80391e0 0x7fd0a803b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:35.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.277+0000 7fd0c92d2700 1 --2- 172.21.15.90:0/4163714777 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fd0a80391e0 0x7fd0a803b6a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0001eb0 tx=0x7fd0c0005de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:05:35.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.282+0000 7fd0ba7fc700 1 -- 172.21.15.90:0/4163714777 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd0b400f8d0 con 0x7fd0c419d290 2024-03-24T12:05:35.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.568+0000 7fd0cb536700 1 -- 172.21.15.90:0/4163714777 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd0c419e110 con 0x7fd0c419d290 2024-03-24T12:05:35.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.568+0000 7fd0ba7fc700 1 -- 172.21.15.90:0/4163714777 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1269 (secure 0 0 0) 0x7fd0b400c150 con 0x7fd0c419d290 2024-03-24T12:05:35.568 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:35 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T12:05:35.568 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:35 smithi090 ceph-mon[33646]: mon.a calling monitor election 2024-03-24T12:05:35.568 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:35 smithi090 ceph-mon[33646]: mon.b calling monitor election 2024-03-24T12:05:35.568 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:35 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:05:35.568 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:35 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:05:35.568 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:35 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:05:35.568 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:35 smithi090 ceph-mon[33646]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:35.568 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:35 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:05:35.569 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:35 smithi090 ceph-mon[33646]: mon.c calling monitor election 2024-03-24T12:05:35.569 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:35 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:05:35.569 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:35 smithi090 ceph-mon[33646]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:35.569 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:35 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:05:35.569 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:35 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:05:35.569 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:35 smithi090 ceph-mon[33646]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-24T12:05:35.569 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:35 smithi090 ceph-mon[33646]: monmap e3: 3 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],b=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0],c=[v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T12:05:35.569 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:35 smithi090 ceph-mon[33646]: fsmap 2024-03-24T12:05:35.569 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:35 smithi090 ceph-mon[33646]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T12:05:35.569 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:35 smithi090 ceph-mon[33646]: mgrmap e13: y(active, since 49s) 2024-03-24T12:05:35.569 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:35 smithi090 ceph-mon[33646]: overall HEALTH_OK 2024-03-24T12:05:35.569 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:35 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:35.569 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:35 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:35.569 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:35 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:35.569 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:35 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:35.570 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-24T12:05:35.570 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":3,"fsid":"66d0ad74-e9d6-11ee-95cd-87774f69a715","modified":"2024-03-24T12:05:30.174802Z","created":"2024-03-24T12:03:39.411497Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3300","nonce":0},{"type":"v1","addr":"172.21.15.6:6789","nonce":0}]},"addr":"172.21.15.6:6789/0","public_addr":"172.21.15.6:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:3301","nonce":0},{"type":"v1","addr":"172.21.15.6:6790","nonce":0}]},"addr":"172.21.15.6:6790/0","public_addr":"172.21.15.6:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-03-24T12:05:35.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.571+0000 7fd0cb536700 1 -- 172.21.15.90:0/4163714777 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fd0a80391e0 msgr2=0x7fd0a803b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:35.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.571+0000 7fd0cb536700 1 --2- 172.21.15.90:0/4163714777 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fd0a80391e0 0x7fd0a803b6a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0001eb0 tx=0x7fd0c0005de0 comp rx=0 tx=0).stop 2024-03-24T12:05:35.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.571+0000 7fd0cb536700 1 -- 172.21.15.90:0/4163714777 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0c419d290 msgr2=0x7fd0c41a1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:35.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.571+0000 7fd0cb536700 1 --2- 172.21.15.90:0/4163714777 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0c419d290 0x7fd0c41a1620 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400b550 tx=0x7fd0b400b910 comp rx=0 tx=0).stop 2024-03-24T12:05:35.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.571+0000 7fd0cb536700 1 -- 172.21.15.90:0/4163714777 shutdown_connections 2024-03-24T12:05:35.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.571+0000 7fd0cb536700 1 --2- 172.21.15.90:0/4163714777 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fd0a80391e0 0x7fd0a803b6a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:35.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.571+0000 7fd0cb536700 1 --2- 172.21.15.90:0/4163714777 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0c41a1b60 0x7fd0c41a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:35.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.571+0000 7fd0cb536700 1 --2- 172.21.15.90:0/4163714777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0b00035c0 0x7fd0c419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:35.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.571+0000 7fd0cb536700 1 --2- 172.21.15.90:0/4163714777 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0c419d290 0x7fd0c41a1620 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:35.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.571+0000 7fd0cb536700 1 -- 172.21.15.90:0/4163714777 >> 172.21.15.90:0/4163714777 conn(0x7fd0c40fc180 msgr2=0x7fd0c4103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:05:35.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.571+0000 7fd0cb536700 1 -- 172.21.15.90:0/4163714777 shutdown_connections 2024-03-24T12:05:35.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:35.571+0000 7fd0cb536700 1 -- 172.21.15.90:0/4163714777 wait complete. 2024-03-24T12:05:35.572 INFO:teuthology.orchestra.run.smithi090.stderr:dumped monmap epoch 3 2024-03-24T12:05:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T12:05:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[30108]: mon.a calling monitor election 2024-03-24T12:05:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[30108]: mon.b calling monitor election 2024-03-24T12:05:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:05:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:05:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:05:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[30108]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:35.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:05:35.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[30108]: mon.c calling monitor election 2024-03-24T12:05:35.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:05:35.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[30108]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:35.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:05:35.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:05:35.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[30108]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-24T12:05:35.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[30108]: monmap e3: 3 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],b=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0],c=[v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T12:05:35.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[30108]: fsmap 2024-03-24T12:05:35.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[30108]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T12:05:35.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[30108]: mgrmap e13: y(active, since 49s) 2024-03-24T12:05:35.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[30108]: overall HEALTH_OK 2024-03-24T12:05:35.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:35.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:35.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:35.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:35.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T12:05:35.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[41734]: mon.a calling monitor election 2024-03-24T12:05:35.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[41734]: mon.b calling monitor election 2024-03-24T12:05:35.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:05:35.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:05:35.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:05:35.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[41734]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:35.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:05:35.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[41734]: mon.c calling monitor election 2024-03-24T12:05:35.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:05:35.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[41734]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:35.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:05:35.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:05:35.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[41734]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-24T12:05:35.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[41734]: monmap e3: 3 mons at {a=[v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0],b=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0],c=[v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-24T12:05:35.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[41734]: fsmap 2024-03-24T12:05:35.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[41734]: osdmap e4: 0 total, 0 up, 0 in 2024-03-24T12:05:35.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[41734]: mgrmap e13: y(active, since 49s) 2024-03-24T12:05:35.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[41734]: overall HEALTH_OK 2024-03-24T12:05:35.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:35.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:35.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:35.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:35 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:36.137 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-24T12:05:36.138 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph config generate-minimal-conf 2024-03-24T12:05:36.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:36 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:05:36.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:36 smithi090 ceph-mon[33646]: from='client.? 172.21.15.90:0/4163714777' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T12:05:36.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:36 smithi090 ceph-mon[33646]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:36.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:36 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:36.369 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:36 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:05:36.418 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:36 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:05:36.418 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:36 smithi006 ceph-mon[30108]: from='client.? 172.21.15.90:0/4163714777' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T12:05:36.418 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:36 smithi006 ceph-mon[30108]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:36.418 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:36 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:36.418 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:36 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:05:36.418 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:36 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:05:36.418 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:36 smithi006 ceph-mon[41734]: from='client.? 172.21.15.90:0/4163714777' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-24T12:05:36.418 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:36 smithi006 ceph-mon[41734]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:36.419 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:36 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:36.419 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:36 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:05:36.727 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:05:38.064 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.064 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.064 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.064 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.064 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:38.064 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:05:38.064 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.064 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.065 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.065 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.065 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.065 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.065 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.065 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.065 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.065 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T12:05:38.065 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T12:05:38.065 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:38.065 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.065 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.065 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.065 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.066 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:38.066 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:05:38.066 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.066 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.066 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.066 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.066 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.066 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.066 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.066 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.066 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.066 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T12:05:38.066 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T12:05:38.066 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:37 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:37 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:37 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:37 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:37 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:37 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:37 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:05:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:37 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:37 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:37 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:37 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:37 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:37 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:37 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:37 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:37 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:37 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T12:05:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:37 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T12:05:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:37 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:38.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.641+0000 7fdf0226e700 1 -- 172.21.15.6:0/4034907745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdefc106600 msgr2=0x7fdefc1069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:38.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.641+0000 7fdf0226e700 1 --2- 172.21.15.6:0/4034907745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdefc106600 0x7fdefc1069e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009910 tx=0x7fdeec009c20 comp rx=0 tx=0).stop 2024-03-24T12:05:38.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.641+0000 7fdf0226e700 1 -- 172.21.15.6:0/4034907745 shutdown_connections 2024-03-24T12:05:38.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.641+0000 7fdf0226e700 1 --2- 172.21.15.6:0/4034907745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdefc106600 0x7fdefc1069e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:38.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.641+0000 7fdf0226e700 1 -- 172.21.15.6:0/4034907745 >> 172.21.15.6:0/4034907745 conn(0x7fdefc0fc020 msgr2=0x7fdefc0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:05:38.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.641+0000 7fdf0226e700 1 -- 172.21.15.6:0/4034907745 shutdown_connections 2024-03-24T12:05:38.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.641+0000 7fdf0226e700 1 -- 172.21.15.6:0/4034907745 wait complete. 2024-03-24T12:05:38.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.642+0000 7fdf0226e700 1 Processor -- start 2024-03-24T12:05:38.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.642+0000 7fdf0226e700 1 -- start start 2024-03-24T12:05:38.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.642+0000 7fdf0226e700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdefc106600 0x7fdefc196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:38.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.642+0000 7fdf0226e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdefc196cc0 0x7fdefc19b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:38.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.642+0000 7fdf0226e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdefc19b540 0x7fdefc19b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:38.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.642+0000 7fdf0226e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc1973b0 con 0x7fdefc19b540 2024-03-24T12:05:38.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.642+0000 7fdf0226e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc1a32b0 con 0x7fdefc196cc0 2024-03-24T12:05:38.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.642+0000 7fdf0226e700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc1a33f0 con 0x7fdefc106600 2024-03-24T12:05:38.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.642+0000 7fdefbfff700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdefc106600 0x7fdefc196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:38.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.642+0000 7fdf0080b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdefc19b540 0x7fdefc19b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:38.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.642+0000 7fdefbfff700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdefc106600 0x7fdefc196780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:38086/0 (socket says 172.21.15.6:38086) 2024-03-24T12:05:38.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.642+0000 7fdefbfff700 1 -- 172.21.15.6:0/1332934736 learned_addr learned my addr 172.21.15.6:0/1332934736 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:05:38.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.642+0000 7fdef3fff700 1 --2- 172.21.15.6:0/1332934736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdefc196cc0 0x7fdefc19b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:38.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.642+0000 7fdefbfff700 1 -- 172.21.15.6:0/1332934736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdefc196cc0 msgr2=0x7fdefc19b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:38.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.642+0000 7fdefbfff700 1 --2- 172.21.15.6:0/1332934736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdefc196cc0 0x7fdefc19b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:38.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.642+0000 7fdefbfff700 1 -- 172.21.15.6:0/1332934736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdefc19b540 msgr2=0x7fdefc19b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:38.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.642+0000 7fdefbfff700 1 --2- 172.21.15.6:0/1332934736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdefc19b540 0x7fdefc19b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:38.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.642+0000 7fdefbfff700 1 -- 172.21.15.6:0/1332934736 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeec0095f0 con 0x7fdefc106600 2024-03-24T12:05:38.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.642+0000 7fdf0080b700 1 --2- 172.21.15.6:0/1332934736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdefc19b540 0x7fdefc19b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:05:38.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.642+0000 7fdefbfff700 1 --2- 172.21.15.6:0/1332934736 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdefc106600 0x7fdefc196780 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00bcf0 tx=0x7fdeec00bd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:05:38.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.643+0000 7fdef9ffb700 1 -- 172.21.15.6:0/1332934736 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec01d070 con 0x7fdefc106600 2024-03-24T12:05:38.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.643+0000 7fdef9ffb700 1 -- 172.21.15.6:0/1332934736 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fdeec005da0 con 0x7fdefc106600 2024-03-24T12:05:38.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.643+0000 7fdf0226e700 1 -- 172.21.15.6:0/1332934736 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdefc1a36d0 con 0x7fdefc106600 2024-03-24T12:05:38.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.643+0000 7fdef9ffb700 1 -- 172.21.15.6:0/1332934736 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec00fa60 con 0x7fdefc106600 2024-03-24T12:05:38.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.643+0000 7fdf0226e700 1 -- 172.21.15.6:0/1332934736 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdefc1a3bd0 con 0x7fdefc106600 2024-03-24T12:05:38.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.643+0000 7fdef9ffb700 1 -- 172.21.15.6:0/1332934736 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7fdeec00fc40 con 0x7fdefc106600 2024-03-24T12:05:38.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.644+0000 7fdef9ffb700 1 --2- 172.21.15.6:0/1332934736 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fdee80391e0 0x7fdee803b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:38.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.644+0000 7fdef9ffb700 1 -- 172.21.15.6:0/1332934736 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdeec04ca80 con 0x7fdefc106600 2024-03-24T12:05:38.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.644+0000 7fdef3fff700 1 --2- 172.21.15.6:0/1332934736 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fdee80391e0 0x7fdee803b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:38.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.644+0000 7fdf0226e700 1 -- 172.21.15.6:0/1332934736 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdefc0632a0 con 0x7fdefc106600 2024-03-24T12:05:38.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.644+0000 7fdef3fff700 1 --2- 172.21.15.6:0/1332934736 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fdee80391e0 0x7fdee803b6a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdee4009850 tx=0x7fdee4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:05:38.648 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.649+0000 7fdef9ffb700 1 -- 172.21.15.6:0/1332934736 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdeec026070 con 0x7fdefc106600 2024-03-24T12:05:38.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.856+0000 7fdf0226e700 1 -- 172.21.15.6:0/1332934736 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fdefc109d90 con 0x7fdefc106600 2024-03-24T12:05:38.856 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.857+0000 7fdef9ffb700 1 -- 172.21.15.6:0/1332934736 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+267 (secure 0 0 0) 0x7fdeec017440 con 0x7fdefc106600 2024-03-24T12:05:38.856 INFO:teuthology.orchestra.run.smithi006.stdout:# minimal ceph.conf for 66d0ad74-e9d6-11ee-95cd-87774f69a715 2024-03-24T12:05:38.856 INFO:teuthology.orchestra.run.smithi006.stdout:[global] 2024-03-24T12:05:38.856 INFO:teuthology.orchestra.run.smithi006.stdout: fsid = 66d0ad74-e9d6-11ee-95cd-87774f69a715 2024-03-24T12:05:38.857 INFO:teuthology.orchestra.run.smithi006.stdout: mon_host = [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] 2024-03-24T12:05:38.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.859+0000 7fdf0226e700 1 -- 172.21.15.6:0/1332934736 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fdee80391e0 msgr2=0x7fdee803b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:38.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.859+0000 7fdf0226e700 1 --2- 172.21.15.6:0/1332934736 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fdee80391e0 0x7fdee803b6a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdee4009850 tx=0x7fdee4008040 comp rx=0 tx=0).stop 2024-03-24T12:05:38.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.859+0000 7fdf0226e700 1 -- 172.21.15.6:0/1332934736 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdefc106600 msgr2=0x7fdefc196780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:38.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.859+0000 7fdf0226e700 1 --2- 172.21.15.6:0/1332934736 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdefc106600 0x7fdefc196780 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00bcf0 tx=0x7fdeec00bd20 comp rx=0 tx=0).stop 2024-03-24T12:05:38.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.859+0000 7fdf0226e700 1 -- 172.21.15.6:0/1332934736 shutdown_connections 2024-03-24T12:05:38.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.859+0000 7fdf0226e700 1 --2- 172.21.15.6:0/1332934736 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fdee80391e0 0x7fdee803b6a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:38.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.859+0000 7fdf0226e700 1 --2- 172.21.15.6:0/1332934736 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdefc19b540 0x7fdefc19b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:38.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.859+0000 7fdf0226e700 1 --2- 172.21.15.6:0/1332934736 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdefc106600 0x7fdefc196780 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:38.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.859+0000 7fdf0226e700 1 --2- 172.21.15.6:0/1332934736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdefc196cc0 0x7fdefc19b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:38.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.859+0000 7fdf0226e700 1 -- 172.21.15.6:0/1332934736 >> 172.21.15.6:0/1332934736 conn(0x7fdefc0fc020 msgr2=0x7fdefc104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:05:38.859 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.859+0000 7fdf0226e700 1 -- 172.21.15.6:0/1332934736 shutdown_connections 2024-03-24T12:05:38.860 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:38.859+0000 7fdf0226e700 1 -- 172.21.15.6:0/1332934736 wait complete. 2024-03-24T12:05:38.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:38 smithi006 ceph-mon[30108]: Updating smithi006:/etc/ceph/ceph.conf 2024-03-24T12:05:38.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:38 smithi006 ceph-mon[30108]: Updating smithi090:/etc/ceph/ceph.conf 2024-03-24T12:05:38.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:38 smithi006 ceph-mon[30108]: Updating smithi090:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T12:05:38.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:38 smithi006 ceph-mon[30108]: Updating smithi006:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T12:05:38.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:38 smithi006 ceph-mon[30108]: Reconfiguring mon.a (unknown last config time)... 2024-03-24T12:05:38.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:38 smithi006 ceph-mon[30108]: Reconfiguring daemon mon.a on smithi006 2024-03-24T12:05:38.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:38 smithi006 ceph-mon[30108]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:38.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:38 smithi006 ceph-mon[41734]: Updating smithi006:/etc/ceph/ceph.conf 2024-03-24T12:05:38.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:38 smithi006 ceph-mon[41734]: Updating smithi090:/etc/ceph/ceph.conf 2024-03-24T12:05:38.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:38 smithi006 ceph-mon[41734]: Updating smithi090:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T12:05:38.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:38 smithi006 ceph-mon[41734]: Updating smithi006:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T12:05:38.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:38 smithi006 ceph-mon[41734]: Reconfiguring mon.a (unknown last config time)... 2024-03-24T12:05:38.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:38 smithi006 ceph-mon[41734]: Reconfiguring daemon mon.a on smithi006 2024-03-24T12:05:38.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:38 smithi006 ceph-mon[41734]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:39.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:38 smithi090 ceph-mon[33646]: Updating smithi006:/etc/ceph/ceph.conf 2024-03-24T12:05:39.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:38 smithi090 ceph-mon[33646]: Updating smithi090:/etc/ceph/ceph.conf 2024-03-24T12:05:39.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:38 smithi090 ceph-mon[33646]: Updating smithi090:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T12:05:39.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:38 smithi090 ceph-mon[33646]: Updating smithi006:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T12:05:39.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:38 smithi090 ceph-mon[33646]: Reconfiguring mon.a (unknown last config time)... 2024-03-24T12:05:39.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:38 smithi090 ceph-mon[33646]: Reconfiguring daemon mon.a on smithi006 2024-03-24T12:05:39.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:38 smithi090 ceph-mon[33646]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:39.543 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-24T12:05:39.543 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-24T12:05:39.543 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/ceph.conf 2024-03-24T12:05:39.573 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-24T12:05:39.573 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-24T12:05:39.642 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-24T12:05:39.642 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.conf 2024-03-24T12:05:39.676 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-24T12:05:39.676 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-24T12:05:39.746 INFO:tasks.cephadm:Adding mgr.y on smithi006 2024-03-24T12:05:39.746 INFO:tasks.cephadm:Adding mgr.x on smithi090 2024-03-24T12:05:39.746 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph orch apply mgr '2;smithi006=y;smithi090=x' 2024-03-24T12:05:39.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:39 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1332934736' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:39.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1332934736' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:39.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:39 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1332934736' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:40.024 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T12:05:40.939 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:40 smithi006 ceph-mon[41734]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:40.939 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:40 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:40.939 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:40 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:40.939 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:40 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T12:05:40.939 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:40 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T12:05:40.939 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:40 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:40.939 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:40 smithi006 ceph-mon[30108]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:40.939 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:40 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:40.940 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:40 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:40.940 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:40 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T12:05:40.940 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:40 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T12:05:40.940 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:40 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:41.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:40 smithi090 ceph-mon[33646]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:41.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:40 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:41.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:40 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:41.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:40 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T12:05:41.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:40 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T12:05:41.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:40 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:41.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.678+0000 7f6b8455d700 1 -- 172.21.15.90:0/4063904611 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b7c1006b0 msgr2=0x7f6b7c100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:41.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.678+0000 7f6b8455d700 1 --2- 172.21.15.90:0/4063904611 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b7c1006b0 0x7f6b7c100ad0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c009910 tx=0x7f6b6c009c20 comp rx=0 tx=0).stop 2024-03-24T12:05:41.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.678+0000 7f6b8455d700 1 -- 172.21.15.90:0/4063904611 shutdown_connections 2024-03-24T12:05:41.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.678+0000 7f6b8455d700 1 --2- 172.21.15.90:0/4063904611 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b7c1006b0 0x7f6b7c100ad0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:41.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.678+0000 7f6b8455d700 1 -- 172.21.15.90:0/4063904611 >> 172.21.15.90:0/4063904611 conn(0x7f6b7c0fbc50 msgr2=0x7f6b7c0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:05:41.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.678+0000 7f6b8455d700 1 -- 172.21.15.90:0/4063904611 shutdown_connections 2024-03-24T12:05:41.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.678+0000 7f6b8455d700 1 -- 172.21.15.90:0/4063904611 wait complete. 2024-03-24T12:05:41.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.679+0000 7f6b8455d700 1 Processor -- start 2024-03-24T12:05:41.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.679+0000 7f6b8455d700 1 -- start start 2024-03-24T12:05:41.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.679+0000 7f6b8455d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b7c1006b0 0x7f6b7c0fdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:41.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.679+0000 7f6b8455d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b7c0fc240 0x7f6b7c0fc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:41.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.679+0000 7f6b8455d700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6b7c0fcc00 0x7f6b7c1a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:41.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.679+0000 7f6b8455d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c1a0920 con 0x7f6b7c1006b0 2024-03-24T12:05:41.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.679+0000 7f6b8455d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c1a0a60 con 0x7f6b7c0fc240 2024-03-24T12:05:41.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.679+0000 7f6b8455d700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c1a0ba0 con 0x7f6b7c0fcc00 2024-03-24T12:05:41.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.679+0000 7f6b81af8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b7c0fc240 0x7f6b7c0fc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:41.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.679+0000 7f6b822f9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b7c1006b0 0x7f6b7c0fdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:41.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.679+0000 7f6b81af8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b7c0fc240 0x7f6b7c0fc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49022/0 (socket says 172.21.15.90:49022) 2024-03-24T12:05:41.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.679+0000 7f6b81af8700 1 -- 172.21.15.90:0/3072843468 learned_addr learned my addr 172.21.15.90:0/3072843468 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-24T12:05:41.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.679+0000 7f6b82afa700 1 --2- 172.21.15.90:0/3072843468 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6b7c0fcc00 0x7f6b7c1a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:41.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.680+0000 7f6b822f9700 1 -- 172.21.15.90:0/3072843468 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6b7c0fcc00 msgr2=0x7f6b7c1a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:41.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.680+0000 7f6b822f9700 1 --2- 172.21.15.90:0/3072843468 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6b7c0fcc00 0x7f6b7c1a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:41.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.680+0000 7f6b822f9700 1 -- 172.21.15.90:0/3072843468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b7c0fc240 msgr2=0x7f6b7c0fc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:41.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.680+0000 7f6b822f9700 1 --2- 172.21.15.90:0/3072843468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b7c0fc240 0x7f6b7c0fc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:41.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.680+0000 7f6b822f9700 1 -- 172.21.15.90:0/3072843468 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c0095f0 con 0x7f6b7c1006b0 2024-03-24T12:05:41.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.680+0000 7f6b81af8700 1 --2- 172.21.15.90:0/3072843468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b7c0fc240 0x7f6b7c0fc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:05:41.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.680+0000 7f6b822f9700 1 --2- 172.21.15.90:0/3072843468 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b7c1006b0 0x7f6b7c0fdbf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c009910 tx=0x7f6b6c004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:05:41.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.680+0000 7f6b737fe700 1 -- 172.21.15.90:0/3072843468 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c00bb60 con 0x7f6b7c1006b0 2024-03-24T12:05:41.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.680+0000 7f6b737fe700 1 -- 172.21.15.90:0/3072843468 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f6b6c005da0 con 0x7f6b7c1006b0 2024-03-24T12:05:41.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.680+0000 7f6b8455d700 1 -- 172.21.15.90:0/3072843468 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b7c1a0e20 con 0x7f6b7c1006b0 2024-03-24T12:05:41.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.680+0000 7f6b737fe700 1 -- 172.21.15.90:0/3072843468 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c00fdf0 con 0x7f6b7c1006b0 2024-03-24T12:05:41.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.680+0000 7f6b8455d700 1 -- 172.21.15.90:0/3072843468 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b7c1a12b0 con 0x7f6b7c1006b0 2024-03-24T12:05:41.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.681+0000 7f6b737fe700 1 -- 172.21.15.90:0/3072843468 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7f6b6c02c440 con 0x7f6b7c1006b0 2024-03-24T12:05:41.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.681+0000 7f6b8455d700 1 -- 172.21.15.90:0/3072843468 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b7c18fef0 con 0x7f6b7c1006b0 2024-03-24T12:05:41.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.681+0000 7f6b737fe700 1 --2- 172.21.15.90:0/3072843468 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f6b680391e0 0x7f6b6803b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:41.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.681+0000 7f6b737fe700 1 -- 172.21.15.90:0/3072843468 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6b6c04cec0 con 0x7f6b7c1006b0 2024-03-24T12:05:41.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.681+0000 7f6b81af8700 1 --2- 172.21.15.90:0/3072843468 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f6b680391e0 0x7f6b6803b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:41.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.682+0000 7f6b81af8700 1 --2- 172.21.15.90:0/3072843468 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f6b680391e0 0x7f6b6803b6a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6b78009f70 tx=0x7f6b780092c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:05:41.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.686+0000 7f6b737fe700 1 -- 172.21.15.90:0/3072843468 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6b6c015940 con 0x7f6b7c1006b0 2024-03-24T12:05:41.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.896+0000 7f6b8455d700 1 -- 172.21.15.90:0/3072843468 --> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi006=y;smithi090=x", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b7c02d7d0 con 0x7f6b680391e0 2024-03-24T12:05:41.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.902+0000 7f6b737fe700 1 -- 172.21.15.90:0/3072843468 <== mgr.14152 v2:172.21.15.6:6800/3847865179 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f6b7c02d7d0 con 0x7f6b680391e0 2024-03-24T12:05:41.902 INFO:teuthology.orchestra.run.smithi090.stdout:Scheduled mgr update... 2024-03-24T12:05:41.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.904+0000 7f6b8455d700 1 -- 172.21.15.90:0/3072843468 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f6b680391e0 msgr2=0x7f6b6803b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:41.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.904+0000 7f6b8455d700 1 --2- 172.21.15.90:0/3072843468 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f6b680391e0 0x7f6b6803b6a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6b78009f70 tx=0x7f6b780092c0 comp rx=0 tx=0).stop 2024-03-24T12:05:41.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.904+0000 7f6b8455d700 1 -- 172.21.15.90:0/3072843468 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b7c1006b0 msgr2=0x7f6b7c0fdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:41.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.904+0000 7f6b8455d700 1 --2- 172.21.15.90:0/3072843468 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b7c1006b0 0x7f6b7c0fdbf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c009910 tx=0x7f6b6c004cf0 comp rx=0 tx=0).stop 2024-03-24T12:05:41.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.904+0000 7f6b8455d700 1 -- 172.21.15.90:0/3072843468 shutdown_connections 2024-03-24T12:05:41.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.904+0000 7f6b8455d700 1 --2- 172.21.15.90:0/3072843468 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f6b680391e0 0x7f6b6803b6a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:41.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.904+0000 7f6b8455d700 1 --2- 172.21.15.90:0/3072843468 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6b7c0fcc00 0x7f6b7c1a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:41.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.904+0000 7f6b8455d700 1 --2- 172.21.15.90:0/3072843468 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6b7c1006b0 0x7f6b7c0fdbf0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:41.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.904+0000 7f6b8455d700 1 --2- 172.21.15.90:0/3072843468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b7c0fc240 0x7f6b7c0fc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:41.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.904+0000 7f6b8455d700 1 -- 172.21.15.90:0/3072843468 >> 172.21.15.90:0/3072843468 conn(0x7f6b7c0fbc50 msgr2=0x7f6b7c069a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:05:41.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.904+0000 7f6b8455d700 1 -- 172.21.15.90:0/3072843468 shutdown_connections 2024-03-24T12:05:41.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:05:41.904+0000 7f6b8455d700 1 -- 172.21.15.90:0/3072843468 wait complete. 2024-03-24T12:05:42.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:41 smithi090 ceph-mon[33646]: Reconfiguring mon.c (monmap changed)... 2024-03-24T12:05:42.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:41 smithi090 ceph-mon[33646]: Reconfiguring daemon mon.c on smithi006 2024-03-24T12:05:42.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:41 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:42.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:41 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:42.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:41 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T12:05:42.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:41 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T12:05:42.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:41 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:42.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:41 smithi006 ceph-mon[30108]: Reconfiguring mon.c (monmap changed)... 2024-03-24T12:05:42.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:41 smithi006 ceph-mon[30108]: Reconfiguring daemon mon.c on smithi006 2024-03-24T12:05:42.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:41 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:42.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:41 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:42.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:41 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T12:05:42.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:41 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T12:05:42.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:41 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:42.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:41 smithi006 ceph-mon[41734]: Reconfiguring mon.c (monmap changed)... 2024-03-24T12:05:42.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:41 smithi006 ceph-mon[41734]: Reconfiguring daemon mon.c on smithi006 2024-03-24T12:05:42.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:41 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:42.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:41 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:42.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:41 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-24T12:05:42.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:41 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-24T12:05:42.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:41 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:42.448 DEBUG:teuthology.orchestra.run.smithi090:mgr.x> sudo journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mgr.x.service 2024-03-24T12:05:42.451 INFO:tasks.cephadm:Deploying OSDs... 2024-03-24T12:05:42.451 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-24T12:05:42.451 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/scratch_devs of=/dev/stdout 2024-03-24T12:05:42.470 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-24T12:05:42.470 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_1 2024-03-24T12:05:42.527 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-24T12:05:42.527 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T12:05:42.527 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-03-24T12:05:42.528 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T12:05:42.528 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T12:05:42.528 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-03-24 12:05:29.621718363 +0000 2024-03-24T12:05:42.528 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-03-24 11:59:23.122339083 +0000 2024-03-24T12:05:42.528 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-03-24 11:59:23.122339083 +0000 2024-03-24T12:05:42.528 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-03-24 11:59:23.122339083 +0000 2024-03-24T12:05:42.528 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-24T12:05:42.598 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-03-24T12:05:42.598 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-03-24T12:05:42.598 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.00028364 s, 1.8 MB/s 2024-03-24T12:05:42.600 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-24T12:05:42.658 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_2 2024-03-24T12:05:42.715 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-24T12:05:42.715 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T12:05:42.715 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-24T12:05:42.716 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T12:05:42.716 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T12:05:42.716 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-03-24 12:05:29.621718363 +0000 2024-03-24T12:05:42.716 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-03-24 11:59:23.458335428 +0000 2024-03-24T12:05:42.716 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-03-24 11:59:23.458335428 +0000 2024-03-24T12:05:42.716 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-03-24 11:59:23.458335428 +0000 2024-03-24T12:05:42.716 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-24T12:05:42.786 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-03-24T12:05:42.786 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-03-24T12:05:42.786 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000309303 s, 1.7 MB/s 2024-03-24T12:05:42.787 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-24T12:05:42.845 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_3 2024-03-24T12:05:42.902 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-24T12:05:42.902 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T12:05:42.902 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-24T12:05:42.902 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T12:05:42.902 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T12:05:42.902 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-03-24 12:05:29.621718363 +0000 2024-03-24T12:05:42.902 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-03-24 11:59:23.809331609 +0000 2024-03-24T12:05:42.902 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-03-24 11:59:23.809331609 +0000 2024-03-24T12:05:42.902 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-03-24 11:59:23.809331609 +0000 2024-03-24T12:05:42.903 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-24T12:05:42.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:42 smithi006 ceph-mon[30108]: Reconfiguring mon.b (monmap changed)... 2024-03-24T12:05:42.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:42 smithi006 ceph-mon[30108]: Reconfiguring daemon mon.b on smithi090 2024-03-24T12:05:42.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:42 smithi006 ceph-mon[30108]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:42.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:42 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:42.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:42 smithi006 ceph-mon[41734]: Reconfiguring mon.b (monmap changed)... 2024-03-24T12:05:42.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:42 smithi006 ceph-mon[41734]: Reconfiguring daemon mon.b on smithi090 2024-03-24T12:05:42.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:42 smithi006 ceph-mon[41734]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:42.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:42 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:42.972 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-03-24T12:05:42.972 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-03-24T12:05:42.972 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000242676 s, 2.1 MB/s 2024-03-24T12:05:42.973 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-24T12:05:43.032 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_4 2024-03-24T12:05:43.088 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-24T12:05:43.088 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T12:05:43.089 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-03-24T12:05:43.089 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T12:05:43.089 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T12:05:43.089 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2024-03-24 12:05:29.622718352 +0000 2024-03-24T12:05:43.089 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2024-03-24 11:59:24.158327812 +0000 2024-03-24T12:05:43.089 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2024-03-24 11:59:24.158327812 +0000 2024-03-24T12:05:43.089 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2024-03-24 11:59:24.158327812 +0000 2024-03-24T12:05:43.089 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-24T12:05:43.158 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2024-03-24T12:05:43.158 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2024-03-24T12:05:43.158 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000223406 s, 2.3 MB/s 2024-03-24T12:05:43.160 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-24T12:05:43.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:42 smithi090 ceph-mon[33646]: Reconfiguring mon.b (monmap changed)... 2024-03-24T12:05:43.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:42 smithi090 ceph-mon[33646]: Reconfiguring daemon mon.b on smithi090 2024-03-24T12:05:43.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:42 smithi090 ceph-mon[33646]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:43.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:42 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:43.217 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-24T12:05:43.218 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/scratch_devs of=/dev/stdout 2024-03-24T12:05:43.234 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-24T12:05:43.234 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_1 2024-03-24T12:05:43.289 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-24T12:05:43.290 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T12:05:43.290 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-03-24T12:05:43.290 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T12:05:43.290 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T12:05:43.290 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-03-24 12:05:20.655828718 +0000 2024-03-24T12:05:43.290 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-03-24 11:59:23.353996700 +0000 2024-03-24T12:05:43.290 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-03-24 11:59:23.353996700 +0000 2024-03-24T12:05:43.290 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-03-24 11:59:23.353996700 +0000 2024-03-24T12:05:43.290 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-24T12:05:43.356 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-03-24T12:05:43.357 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-03-24T12:05:43.357 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000209898 s, 2.4 MB/s 2024-03-24T12:05:43.358 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-24T12:05:43.414 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_2 2024-03-24T12:05:43.469 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-24T12:05:43.470 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T12:05:43.470 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-24T12:05:43.470 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T12:05:43.470 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T12:05:43.470 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-03-24 12:05:20.656828701 +0000 2024-03-24T12:05:43.470 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-03-24 11:59:23.687990922 +0000 2024-03-24T12:05:43.470 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-03-24 11:59:23.687990922 +0000 2024-03-24T12:05:43.470 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-03-24 11:59:23.687990922 +0000 2024-03-24T12:05:43.470 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-24T12:05:43.537 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-03-24T12:05:43.538 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-03-24T12:05:43.538 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000205766 s, 2.5 MB/s 2024-03-24T12:05:43.539 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-24T12:05:43.595 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_3 2024-03-24T12:05:43.649 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-24T12:05:43.649 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T12:05:43.649 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-03-24T12:05:43.649 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T12:05:43.649 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T12:05:43.650 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-03-24 12:05:20.656828701 +0000 2024-03-24T12:05:43.650 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-03-24 11:59:24.020985161 +0000 2024-03-24T12:05:43.650 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-03-24 11:59:24.020985161 +0000 2024-03-24T12:05:43.650 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-03-24 11:59:24.020985161 +0000 2024-03-24T12:05:43.650 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-24T12:05:43.716 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-03-24T12:05:43.716 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-03-24T12:05:43.716 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000179205 s, 2.9 MB/s 2024-03-24T12:05:43.717 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-24T12:05:43.772 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_4 2024-03-24T12:05:43.827 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-24T12:05:43.827 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-24T12:05:43.827 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-03-24T12:05:43.827 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-24T12:05:43.827 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-03-24T12:05:43.827 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-03-24 12:05:20.656828701 +0000 2024-03-24T12:05:43.827 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-03-24 11:59:24.359979297 +0000 2024-03-24T12:05:43.827 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-03-24 11:59:24.359979297 +0000 2024-03-24T12:05:43.828 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-03-24 11:59:24.359979297 +0000 2024-03-24T12:05:43.828 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-24T12:05:43.893 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-03-24T12:05:43.893 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-03-24T12:05:43.893 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.00020201 s, 2.5 MB/s 2024-03-24T12:05:43.894 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-24T12:05:43.949 INFO:tasks.cephadm:Deploying osd.0 on smithi006 with /dev/vg_nvme/lv_4... 2024-03-24T12:05:43.949 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-24T12:05:44.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:43 smithi090 ceph-mon[33646]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi006=y;smithi090=x", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:05:44.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:43 smithi090 ceph-mon[33646]: Saving service mgr spec with placement smithi006=y;smithi090=x;count:2 2024-03-24T12:05:44.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:43 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:44.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:43 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:44.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:43 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:05:44.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:43 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:44.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:43 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:05:44.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:43 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:44.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:43 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T12:05:44.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:43 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-24T12:05:44.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:43 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T12:05:44.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:43 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:44.196 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:05:44.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[30108]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi006=y;smithi090=x", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:05:44.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[30108]: Saving service mgr spec with placement smithi006=y;smithi090=x;count:2 2024-03-24T12:05:44.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:44.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:44.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:05:44.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:44.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:05:44.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:44.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T12:05:44.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-24T12:05:44.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T12:05:44.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:44.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[41734]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi006=y;smithi090=x", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:05:44.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[41734]: Saving service mgr spec with placement smithi006=y;smithi090=x;count:2 2024-03-24T12:05:44.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:44.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:44.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:05:44.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:44.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:05:44.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:44.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T12:05:44.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-24T12:05:44.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T12:05:44.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:43 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:44.899 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:44 smithi090 ceph-mon[33646]: Deploying daemon mgr.x on smithi090 2024-03-24T12:05:44.899 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:44 smithi090 ceph-mon[33646]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:44.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:44 smithi006 ceph-mon[30108]: Deploying daemon mgr.x on smithi090 2024-03-24T12:05:44.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:44 smithi006 ceph-mon[30108]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:44.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:44 smithi006 ceph-mon[41734]: Deploying daemon mgr.x on smithi090 2024-03-24T12:05:44.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:44 smithi006 ceph-mon[41734]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:45.895 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:45 smithi090 podman[35701]: 2024-03-24 12:05:45.788890497 +0000 UTC m=+0.118275897 container create 4b38178b7a2effbe65b9411baabd5bc9120c8d596ead15978e8d63fdf30d0961 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD) 2024-03-24T12:05:46.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:45 smithi090 ceph-mon[33646]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:46.178 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:46 smithi090 podman[35701]: 2024-03-24 12:05:46.017140399 +0000 UTC m=+0.346525813 container init 4b38178b7a2effbe65b9411baabd5bc9120c8d596ead15978e8d63fdf30d0961 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-24T12:05:46.178 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:46 smithi090 podman[35701]: 2024-03-24 12:05:46.022603652 +0000 UTC m=+0.351989053 container start 4b38178b7a2effbe65b9411baabd5bc9120c8d596ead15978e8d63fdf30d0961 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T12:05:46.179 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:46 smithi090 ceph-mgr[35739]: -- 172.21.15.90:0/2605210760 <== mon.1 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563f366661a0 con 0x563f3663cc00 2024-03-24T12:05:46.179 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:46 smithi090 bash[35701]: 4b38178b7a2effbe65b9411baabd5bc9120c8d596ead15978e8d63fdf30d0961 2024-03-24T12:05:46.179 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:46 smithi090 systemd[1]: Started Ceph mgr.x for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:05:46.186 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:45 smithi006 ceph-mon[30108]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:46.187 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:45 smithi006 ceph-mon[41734]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:46.187 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-24T12:05:46.187 INFO:teuthology.orchestra.run.smithi006.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-24T12:05:46.187 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10+0 records in 2024-03-24T12:05:46.187 INFO:teuthology.orchestra.run.smithi006.stderr:10+0 records out 2024-03-24T12:05:46.187 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111261 s, 942 MB/s 2024-03-24T12:05:46.187 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping successful for: 2024-03-24T12:05:46.792 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi006:vg_nvme/lv_4 2024-03-24T12:05:46.928 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:46 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:46.577+0000 7f515e3b3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T12:05:46.928 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:46 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:46.673+0000 7f515e3b3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T12:05:47.038 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:05:47.353 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:47 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:47.354 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:47 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:47.354 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:47 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:47.354 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:47 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:47.354 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:47 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:05:47.408 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:47 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:47.408 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:47 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:47.408 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:47 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:47.408 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:47 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:47.408 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:47 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:05:47.409 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:47 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:47.409 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:47 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:47.409 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:47 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:47.409 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:47 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:47.409 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:47 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:05:47.678 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:47 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:47.353+0000 7f515e3b3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T12:05:47.678 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:47 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:47.446+0000 7f515e3b3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T12:05:47.678 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:47 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:47.595+0000 7f515e3b3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T12:05:48.037 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:47 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:47.694+0000 7f515e3b3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T12:05:48.038 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:47 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:47.776+0000 7f515e3b3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T12:05:48.038 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:48 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:48.038+0000 7f515e3b3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-24T12:05:48.324 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:48 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:48.146+0000 7f515e3b3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T12:05:48.324 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:48 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:48.230+0000 7f515e3b3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T12:05:48.324 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:48 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:48.324+0000 7f515e3b3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T12:05:48.678 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:48 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:48.498+0000 7f515e3b3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T12:05:48.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:48 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:48.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:48 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:48.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:48 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:48.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:48 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:05:48.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:48 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:48.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:48 smithi090 ceph-mon[33646]: Reconfiguring mgr.y (unknown last config time)... 2024-03-24T12:05:48.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:48 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T12:05:48.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:48 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T12:05:48.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:48 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:48.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:48 smithi090 ceph-mon[33646]: Reconfiguring daemon mgr.y on smithi006 2024-03-24T12:05:48.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:48 smithi090 ceph-mon[33646]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:48.767 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:48 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:48.767 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:48 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:48.767 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:48 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:48.767 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:48 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:05:48.768 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:48 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:48.768 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:48 smithi006 ceph-mon[30108]: Reconfiguring mgr.y (unknown last config time)... 2024-03-24T12:05:48.768 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:48 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T12:05:48.768 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:48 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T12:05:48.768 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:48 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:48.768 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:48 smithi006 ceph-mon[30108]: Reconfiguring daemon mgr.y on smithi006 2024-03-24T12:05:48.768 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:48 smithi006 ceph-mon[30108]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:48.768 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:48 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:48.768 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:48 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:48.768 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:48 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:48.768 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:48 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:05:48.768 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:48 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:48.768 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:48 smithi006 ceph-mon[41734]: Reconfiguring mgr.y (unknown last config time)... 2024-03-24T12:05:48.768 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:48 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-24T12:05:48.769 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:48 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-24T12:05:48.769 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:48 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:48.769 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:48 smithi006 ceph-mon[41734]: Reconfiguring daemon mgr.y on smithi006 2024-03-24T12:05:48.769 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:48 smithi006 ceph-mon[41734]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:49.556 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.557+0000 7f6368c10700 1 -- 172.21.15.6:0/3784384214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63641005a0 msgr2=0x7f63641009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:49.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.557+0000 7f6368c10700 1 --2- 172.21.15.6:0/3784384214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63641005a0 0x7f63641009c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6354009990 tx=0x7f6354009ca0 comp rx=0 tx=0).stop 2024-03-24T12:05:49.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.557+0000 7f6368c10700 1 -- 172.21.15.6:0/3784384214 shutdown_connections 2024-03-24T12:05:49.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.557+0000 7f6368c10700 1 --2- 172.21.15.6:0/3784384214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63641018c0 0x7f6364101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:49.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.557+0000 7f6368c10700 1 --2- 172.21.15.6:0/3784384214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63641005a0 0x7f63641009c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:49.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.557+0000 7f6368c10700 1 --2- 172.21.15.6:0/3784384214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6364100f00 0x7f6364101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:49.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.557+0000 7f6368c10700 1 -- 172.21.15.6:0/3784384214 >> 172.21.15.6:0/3784384214 conn(0x7f63640fc140 msgr2=0x7f63640fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:05:49.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.557+0000 7f6368c10700 1 -- 172.21.15.6:0/3784384214 shutdown_connections 2024-03-24T12:05:49.557 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.557+0000 7f6368c10700 1 -- 172.21.15.6:0/3784384214 wait complete. 2024-03-24T12:05:49.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.558+0000 7f6368c10700 1 Processor -- start 2024-03-24T12:05:49.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.558+0000 7f6368c10700 1 -- start start 2024-03-24T12:05:49.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.558+0000 7f6368c10700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63641005a0 0x7f6364199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:49.559 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.558+0000 7f6368c10700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6364100f00 0x7f6364199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:49.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.558+0000 7f6368c10700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63641018c0 0x7f63641a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:49.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.558+0000 7f6368c10700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63641a0f90 con 0x7f63641005a0 2024-03-24T12:05:49.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.558+0000 7f6368c10700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63641a10d0 con 0x7f6364100f00 2024-03-24T12:05:49.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.558+0000 7f6368c10700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63641a1260 con 0x7f63641018c0 2024-03-24T12:05:49.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.558+0000 7f636259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63641005a0 0x7f6364199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:49.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.558+0000 7f636259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63641005a0 0x7f6364199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37716/0 (socket says 172.21.15.6:37716) 2024-03-24T12:05:49.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.558+0000 7f636259c700 1 -- 172.21.15.6:0/2710132335 learned_addr learned my addr 172.21.15.6:0/2710132335 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:05:49.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.558+0000 7f6361d9b700 1 --2- 172.21.15.6:0/2710132335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6364100f00 0x7f6364199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:49.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.558+0000 7f6362d9d700 1 --2- 172.21.15.6:0/2710132335 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63641018c0 0x7f63641a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:49.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.558+0000 7f636259c700 1 -- 172.21.15.6:0/2710132335 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63641018c0 msgr2=0x7f63641a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:49.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.558+0000 7f636259c700 1 --2- 172.21.15.6:0/2710132335 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63641018c0 0x7f63641a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:49.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.558+0000 7f636259c700 1 -- 172.21.15.6:0/2710132335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6364100f00 msgr2=0x7f6364199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:05:49.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.558+0000 7f636259c700 1 --2- 172.21.15.6:0/2710132335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6364100f00 0x7f6364199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:05:49.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.558+0000 7f636259c700 1 -- 172.21.15.6:0/2710132335 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6358009520 con 0x7f63641005a0 2024-03-24T12:05:49.560 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.558+0000 7f6362d9d700 1 --2- 172.21.15.6:0/2710132335 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63641018c0 0x7f63641a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T12:05:49.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.558+0000 7f6361d9b700 1 --2- 172.21.15.6:0/2710132335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6364100f00 0x7f6364199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:05:49.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.559+0000 7f636259c700 1 --2- 172.21.15.6:0/2710132335 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63641005a0 0x7f6364199400 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6354003f50 tx=0x7f6354004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:05:49.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.559+0000 7f63537fe700 1 -- 172.21.15.6:0/2710132335 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635401d070 con 0x7f63641005a0 2024-03-24T12:05:49.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.559+0000 7f63537fe700 1 -- 172.21.15.6:0/2710132335 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f635400bd90 con 0x7f63641005a0 2024-03-24T12:05:49.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.559+0000 7f6368c10700 1 -- 172.21.15.6:0/2710132335 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63540095f0 con 0x7f63641005a0 2024-03-24T12:05:49.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.559+0000 7f63537fe700 1 -- 172.21.15.6:0/2710132335 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354005280 con 0x7f63641005a0 2024-03-24T12:05:49.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.559+0000 7f6368c10700 1 -- 172.21.15.6:0/2710132335 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63641a1700 con 0x7f63641005a0 2024-03-24T12:05:49.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.560+0000 7f63537fe700 1 -- 172.21.15.6:0/2710132335 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 13) v1 ==== 45824+0+0 (secure 0 0 0) 0x7f63540053e0 con 0x7f63641005a0 2024-03-24T12:05:49.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.560+0000 7f63537fe700 1 --2- 172.21.15.6:0/2710132335 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f634c038f90 0x7f634c03b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:05:49.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.560+0000 7f6368c10700 1 -- 172.21.15.6:0/2710132335 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f636410b380 con 0x7f63641005a0 2024-03-24T12:05:49.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.560+0000 7f63537fe700 1 -- 172.21.15.6:0/2710132335 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f635404cc00 con 0x7f63641005a0 2024-03-24T12:05:49.561 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.560+0000 7f6361d9b700 1 --2- 172.21.15.6:0/2710132335 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f634c038f90 0x7f634c03b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:05:49.562 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.560+0000 7f6361d9b700 1 --2- 172.21.15.6:0/2710132335 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f634c038f90 0x7f634c03b450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6358009fd0 tx=0x7f6358009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:05:49.564 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.565+0000 7f63537fe700 1 -- 172.21.15.6:0/2710132335 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6354020370 con 0x7f63641005a0 2024-03-24T12:05:49.778 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:49.778+0000 7f6368c10700 1 -- 172.21.15.6:0/2710132335 --> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f63640620b0 con 0x7f634c038f90 2024-03-24T12:05:49.928 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:49 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:49.462+0000 7f515e3b3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T12:05:49.928 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:49 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:49.594+0000 7f515e3b3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T12:05:50.232 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:49 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:49.946+0000 7f515e3b3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T12:05:50.233 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:50 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:50.039+0000 7f515e3b3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T12:05:50.233 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:50 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:50.233+0000 7f515e3b3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T12:05:50.674 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:50 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:50.675 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:50 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:50.675 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:50 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:05:50.675 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:50 smithi006 ceph-mon[41734]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:05:50.675 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:50 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:05:50.675 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:50 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:05:50.675 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:50 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:50.675 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:50 smithi006 ceph-mon[41734]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:50.675 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:50 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:50.675 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:50 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:05:50.675 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:50 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:50.675 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:50 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:50.675 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:50 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:50.675 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:50 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:05:50.676 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:50 smithi006 ceph-mon[30108]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:05:50.676 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:50 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:05:50.676 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:50 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:05:50.676 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:50 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:50.676 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:50 smithi006 ceph-mon[30108]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:50.676 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:50 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:50.676 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:50 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:05:50.676 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:50 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:50.678 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:50 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:50.415+0000 7f515e3b3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T12:05:50.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:50 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:50.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:50 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:50.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:50 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:05:50.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:50 smithi090 ceph-mon[33646]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:05:50.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:50 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:05:50.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:50 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:05:50.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:50 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:50.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:50 smithi090 ceph-mon[33646]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:50.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:50 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:50.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:50 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:05:50.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:50 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:50.956 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:50 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:50.699+0000 7f515e3b3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T12:05:50.956 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:50 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:50.956+0000 7f515e3b3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T12:05:51.428 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:51 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:51.190+0000 7f515e3b3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T12:05:51.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:51 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:51.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:51 smithi006 ceph-mon[41734]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:51.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:51 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:51.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:51 smithi006 ceph-mon[30108]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:52.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:51 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:05:52.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:51 smithi090 ceph-mon[33646]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:52.178 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:51 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:51.980+0000 7f515e3b3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T12:05:52.856 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:52.544+0000 7f515e3b3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T12:05:52.856 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:52.642+0000 7f515e3b3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T12:05:52.856 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:05:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:05:52.856+0000 7f515e3b3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T12:05:53.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:52 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3890604566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3542a64b-e3c1-4a04-ba1c-bdfbcd4a0aa4"}]: dispatch 2024-03-24T12:05:53.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:52 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3890604566' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3542a64b-e3c1-4a04-ba1c-bdfbcd4a0aa4"}]': finished 2024-03-24T12:05:53.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:52 smithi090 ceph-mon[33646]: osdmap e5: 1 total, 0 up, 1 in 2024-03-24T12:05:53.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:52 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T12:05:53.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:52 smithi090 ceph-mon[33646]: Standby manager daemon x started 2024-03-24T12:05:53.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:52 smithi090 ceph-mon[33646]: from='mgr.? 172.21.15.90:0/612149637' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T12:05:53.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:52 smithi090 ceph-mon[33646]: from='mgr.? 172.21.15.90:0/612149637' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T12:05:53.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:52 smithi090 ceph-mon[33646]: from='mgr.? 172.21.15.90:0/612149637' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T12:05:53.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:52 smithi090 ceph-mon[33646]: from='mgr.? 172.21.15.90:0/612149637' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T12:05:53.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:52 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3890604566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3542a64b-e3c1-4a04-ba1c-bdfbcd4a0aa4"}]: dispatch 2024-03-24T12:05:53.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:52 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3890604566' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3542a64b-e3c1-4a04-ba1c-bdfbcd4a0aa4"}]': finished 2024-03-24T12:05:53.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:52 smithi006 ceph-mon[41734]: osdmap e5: 1 total, 0 up, 1 in 2024-03-24T12:05:53.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:52 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T12:05:53.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:52 smithi006 ceph-mon[41734]: Standby manager daemon x started 2024-03-24T12:05:53.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:52 smithi006 ceph-mon[41734]: from='mgr.? 172.21.15.90:0/612149637' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T12:05:53.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:52 smithi006 ceph-mon[41734]: from='mgr.? 172.21.15.90:0/612149637' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T12:05:53.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:52 smithi006 ceph-mon[41734]: from='mgr.? 172.21.15.90:0/612149637' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T12:05:53.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:52 smithi006 ceph-mon[41734]: from='mgr.? 172.21.15.90:0/612149637' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T12:05:53.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:52 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3890604566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3542a64b-e3c1-4a04-ba1c-bdfbcd4a0aa4"}]: dispatch 2024-03-24T12:05:53.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:52 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3890604566' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3542a64b-e3c1-4a04-ba1c-bdfbcd4a0aa4"}]': finished 2024-03-24T12:05:53.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:52 smithi006 ceph-mon[30108]: osdmap e5: 1 total, 0 up, 1 in 2024-03-24T12:05:53.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:52 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T12:05:53.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:52 smithi006 ceph-mon[30108]: Standby manager daemon x started 2024-03-24T12:05:53.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:52 smithi006 ceph-mon[30108]: from='mgr.? 172.21.15.90:0/612149637' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T12:05:53.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:52 smithi006 ceph-mon[30108]: from='mgr.? 172.21.15.90:0/612149637' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T12:05:53.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:52 smithi006 ceph-mon[30108]: from='mgr.? 172.21.15.90:0/612149637' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T12:05:53.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:52 smithi006 ceph-mon[30108]: from='mgr.? 172.21.15.90:0/612149637' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T12:05:53.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:05:53.547+0000 7f63537fe700 1 -- 172.21.15.6:0/2710132335 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 14) v1 ==== 91023+0+0 (secure 0 0 0) 0x7f6354022990 con 0x7f63641005a0 2024-03-24T12:05:54.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:53 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1813096420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:05:54.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:53 smithi090 ceph-mon[33646]: mgrmap e14: y(active, since 67s), standbys: x 2024-03-24T12:05:54.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:53 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T12:05:54.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:53 smithi090 ceph-mon[33646]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:54.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:53 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1813096420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:05:54.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:53 smithi006 ceph-mon[41734]: mgrmap e14: y(active, since 67s), standbys: x 2024-03-24T12:05:54.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:53 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T12:05:54.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:53 smithi006 ceph-mon[41734]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:54.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:53 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1813096420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:05:54.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:53 smithi006 ceph-mon[30108]: mgrmap e14: y(active, since 67s), standbys: x 2024-03-24T12:05:54.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:53 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T12:05:54.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:53 smithi006 ceph-mon[30108]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:56.136 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:55 smithi006 ceph-mon[30108]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:56.137 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:55 smithi006 ceph-mon[41734]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:56.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:55 smithi090 ceph-mon[33646]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:58.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:57 smithi090 ceph-mon[33646]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:58.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:57 smithi006 ceph-mon[30108]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:58.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:57 smithi006 ceph-mon[41734]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:05:59.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:59 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T12:05:59.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:05:59 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:59.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:59 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T12:05:59.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:05:59 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:05:59.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:59 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-24T12:05:59.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:05:59 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:00.355 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:00 smithi006 ceph-mon[30108]: Deploying daemon osd.0 on smithi006 2024-03-24T12:06:00.356 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:00 smithi006 ceph-mon[30108]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:06:00.356 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:00 smithi006 ceph-mon[41734]: Deploying daemon osd.0 on smithi006 2024-03-24T12:06:00.356 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:00 smithi006 ceph-mon[41734]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:06:00.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:00 smithi090 ceph-mon[33646]: Deploying daemon osd.0 on smithi006 2024-03-24T12:06:00.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:00 smithi090 ceph-mon[33646]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:06:02.020 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:01 smithi006 ceph-mon[30108]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:06:02.021 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:01 smithi006 ceph-mon[41734]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:06:02.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:01 smithi090 ceph-mon[33646]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:06:04.072 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:03 smithi006 ceph-mon[30108]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:06:04.072 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:03 smithi006 ceph-mon[41734]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:06:04.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:03 smithi090 ceph-mon[33646]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:06:05.889 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:05 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:05.889 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:05 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:05.889 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:05 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:06:05.889 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:05 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:05.889 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:05 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:05.890 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:05 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:06:05.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:05 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:05.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:05 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:05.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:05 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:06:06.836 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:06 smithi006 ceph-mon[30108]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:06:06.836 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:06 smithi006 ceph-mon[41734]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:06:06.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:06 smithi090 ceph-mon[33646]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:06:08.025 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:07 smithi006 ceph-mon[30108]: from='osd.0 [v2:172.21.15.6:6802/195441355,v1:172.21.15.6:6803/195441355]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T12:06:08.026 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:07 smithi006 ceph-mon[41734]: from='osd.0 [v2:172.21.15.6:6802/195441355,v1:172.21.15.6:6803/195441355]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T12:06:08.081 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:08.081+0000 7f63537fe700 1 -- 172.21.15.6:0/2710132335 <== mgr.14152 v2:172.21.15.6:6800/3847865179 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f63640620b0 con 0x7f634c038f90 2024-03-24T12:06:08.081 INFO:teuthology.orchestra.run.smithi006.stdout:Created osd(s) 0 on host 'smithi006' 2024-03-24T12:06:08.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:08.083+0000 7f6368c10700 1 -- 172.21.15.6:0/2710132335 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f634c038f90 msgr2=0x7f634c03b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:06:08.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:08.083+0000 7f6368c10700 1 --2- 172.21.15.6:0/2710132335 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f634c038f90 0x7f634c03b450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6358009fd0 tx=0x7f6358009260 comp rx=0 tx=0).stop 2024-03-24T12:06:08.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:08.083+0000 7f6368c10700 1 -- 172.21.15.6:0/2710132335 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63641005a0 msgr2=0x7f6364199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:06:08.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:08.083+0000 7f6368c10700 1 --2- 172.21.15.6:0/2710132335 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63641005a0 0x7f6364199400 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6354003f50 tx=0x7f6354004730 comp rx=0 tx=0).stop 2024-03-24T12:06:08.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:08.083+0000 7f6368c10700 1 -- 172.21.15.6:0/2710132335 shutdown_connections 2024-03-24T12:06:08.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:08.083+0000 7f6368c10700 1 --2- 172.21.15.6:0/2710132335 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f634c038f90 0x7f634c03b450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:06:08.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:08.083+0000 7f6368c10700 1 --2- 172.21.15.6:0/2710132335 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63641018c0 0x7f63641a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:06:08.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:08.083+0000 7f6368c10700 1 --2- 172.21.15.6:0/2710132335 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63641005a0 0x7f6364199400 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:06:08.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:08.083+0000 7f6368c10700 1 --2- 172.21.15.6:0/2710132335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6364100f00 0x7f6364199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:06:08.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:08.083+0000 7f6368c10700 1 -- 172.21.15.6:0/2710132335 >> 172.21.15.6:0/2710132335 conn(0x7f63640fc140 msgr2=0x7f63640fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:06:08.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:08.083+0000 7f6368c10700 1 -- 172.21.15.6:0/2710132335 shutdown_connections 2024-03-24T12:06:08.083 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:08.083+0000 7f6368c10700 1 -- 172.21.15.6:0/2710132335 wait complete. 2024-03-24T12:06:08.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:07 smithi090 ceph-mon[33646]: from='osd.0 [v2:172.21.15.6:6802/195441355,v1:172.21.15.6:6803/195441355]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-24T12:06:08.697 DEBUG:teuthology.orchestra.run.smithi006:osd.0> sudo journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.0.service 2024-03-24T12:06:08.699 INFO:tasks.cephadm:Deploying osd.1 on smithi006 with /dev/vg_nvme/lv_3... 2024-03-24T12:06:08.699 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-24T12:06:08.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:08 smithi006 ceph-mon[30108]: from='osd.0 [v2:172.21.15.6:6802/195441355,v1:172.21.15.6:6803/195441355]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-24T12:06:08.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:08 smithi006 ceph-mon[30108]: osdmap e6: 1 total, 0 up, 1 in 2024-03-24T12:06:08.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:08 smithi006 ceph-mon[30108]: from='osd.0 [v2:172.21.15.6:6802/195441355,v1:172.21.15.6:6803/195441355]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-24T12:06:08.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:08 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T12:06:08.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:08 smithi006 ceph-mon[30108]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:06:08.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:08 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:08.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:08 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:08.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:08 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:08.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:08 smithi006 ceph-mon[30108]: from='osd.0 [v2:172.21.15.6:6802/195441355,v1:172.21.15.6:6803/195441355]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-03-24T12:06:08.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:08 smithi006 ceph-mon[41734]: from='osd.0 [v2:172.21.15.6:6802/195441355,v1:172.21.15.6:6803/195441355]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-24T12:06:08.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:08 smithi006 ceph-mon[41734]: osdmap e6: 1 total, 0 up, 1 in 2024-03-24T12:06:08.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:08 smithi006 ceph-mon[41734]: from='osd.0 [v2:172.21.15.6:6802/195441355,v1:172.21.15.6:6803/195441355]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-24T12:06:08.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:08 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T12:06:08.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:08 smithi006 ceph-mon[41734]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:06:08.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:08 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:08.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:08 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:08.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:08 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:08.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:08 smithi006 ceph-mon[41734]: from='osd.0 [v2:172.21.15.6:6802/195441355,v1:172.21.15.6:6803/195441355]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-03-24T12:06:08.963 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:06:08 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-0[46125]: 2024-03-24T12:06:08.827+0000 7ff5e775c700 -1 osd.0 0 waiting for initial osdmap 2024-03-24T12:06:08.963 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:06:08 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-0[46125]: 2024-03-24T12:06:08.831+0000 7ff5e354f700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T12:06:09.000 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:06:09.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:08 smithi090 ceph-mon[33646]: from='osd.0 [v2:172.21.15.6:6802/195441355,v1:172.21.15.6:6803/195441355]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-24T12:06:09.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:08 smithi090 ceph-mon[33646]: osdmap e6: 1 total, 0 up, 1 in 2024-03-24T12:06:09.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:08 smithi090 ceph-mon[33646]: from='osd.0 [v2:172.21.15.6:6802/195441355,v1:172.21.15.6:6803/195441355]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-24T12:06:09.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:08 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T12:06:09.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:08 smithi090 ceph-mon[33646]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:06:09.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:08 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:09.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:08 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:09.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:08 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:09.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:08 smithi090 ceph-mon[33646]: from='osd.0 [v2:172.21.15.6:6802/195441355,v1:172.21.15.6:6803/195441355]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-03-24T12:06:09.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:09 smithi006 ceph-mon[30108]: osdmap e7: 1 total, 0 up, 1 in 2024-03-24T12:06:09.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:09 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T12:06:09.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:09 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:09.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:09 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T12:06:09.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:09 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T12:06:09.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:09 smithi006 ceph-mon[41734]: osdmap e7: 1 total, 0 up, 1 in 2024-03-24T12:06:09.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:09 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T12:06:09.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:09 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:09.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:09 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T12:06:09.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:09 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T12:06:10.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:09 smithi090 ceph-mon[33646]: osdmap e7: 1 total, 0 up, 1 in 2024-03-24T12:06:10.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:09 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T12:06:10.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:09 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:10.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:09 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T12:06:10.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:09 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T12:06:10.873 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-24T12:06:10.873 INFO:teuthology.orchestra.run.smithi006.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-24T12:06:10.873 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10+0 records in 2024-03-24T12:06:10.874 INFO:teuthology.orchestra.run.smithi006.stderr:10+0 records out 2024-03-24T12:06:10.874 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0155234 s, 675 MB/s 2024-03-24T12:06:10.874 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping successful for: 2024-03-24T12:06:11.126 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:10 smithi006 ceph-mon[30108]: purged_snaps scrub starts 2024-03-24T12:06:11.126 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:10 smithi006 ceph-mon[30108]: purged_snaps scrub ok 2024-03-24T12:06:11.126 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:10 smithi006 ceph-mon[30108]: osd.0 [v2:172.21.15.6:6802/195441355,v1:172.21.15.6:6803/195441355] boot 2024-03-24T12:06:11.126 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:10 smithi006 ceph-mon[30108]: osdmap e8: 1 total, 1 up, 1 in 2024-03-24T12:06:11.126 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:10 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T12:06:11.127 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:10 smithi006 ceph-mon[30108]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:06:11.127 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:10 smithi006 ceph-mon[41734]: purged_snaps scrub starts 2024-03-24T12:06:11.127 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:10 smithi006 ceph-mon[41734]: purged_snaps scrub ok 2024-03-24T12:06:11.127 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:10 smithi006 ceph-mon[41734]: osd.0 [v2:172.21.15.6:6802/195441355,v1:172.21.15.6:6803/195441355] boot 2024-03-24T12:06:11.127 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:10 smithi006 ceph-mon[41734]: osdmap e8: 1 total, 1 up, 1 in 2024-03-24T12:06:11.127 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:10 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T12:06:11.127 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:10 smithi006 ceph-mon[41734]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:06:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:10 smithi090 ceph-mon[33646]: purged_snaps scrub starts 2024-03-24T12:06:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:10 smithi090 ceph-mon[33646]: purged_snaps scrub ok 2024-03-24T12:06:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:10 smithi090 ceph-mon[33646]: osd.0 [v2:172.21.15.6:6802/195441355,v1:172.21.15.6:6803/195441355] boot 2024-03-24T12:06:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:10 smithi090 ceph-mon[33646]: osdmap e8: 1 total, 1 up, 1 in 2024-03-24T12:06:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:10 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T12:06:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:10 smithi090 ceph-mon[33646]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-24T12:06:11.711 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi006:vg_nvme/lv_3 2024-03-24T12:06:11.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:11 smithi006 ceph-mon[30108]: osdmap e9: 1 total, 1 up, 1 in 2024-03-24T12:06:11.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:11 smithi006 ceph-mon[41734]: osdmap e9: 1 total, 1 up, 1 in 2024-03-24T12:06:12.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:11 smithi090 ceph-mon[33646]: osdmap e9: 1 total, 1 up, 1 in 2024-03-24T12:06:12.542 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:06:12.908 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:12 smithi006 ceph-mon[30108]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:12.908 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:12 smithi006 ceph-mon[41734]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:13.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:12 smithi090 ceph-mon[33646]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:14.539 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.538+0000 7f850c53d700 1 -- 172.21.15.6:0/3839951976 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8504101e10 msgr2=0x7f8504102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:06:14.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.538+0000 7f850c53d700 1 --2- 172.21.15.6:0/3839951976 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8504101e10 0x7f8504102270 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009920 tx=0x7f84f8009c30 comp rx=0 tx=0).stop 2024-03-24T12:06:14.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.538+0000 7f850c53d700 1 -- 172.21.15.6:0/3839951976 shutdown_connections 2024-03-24T12:06:14.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.538+0000 7f850c53d700 1 --2- 172.21.15.6:0/3839951976 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85041027b0 0x7f8504109060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:06:14.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.538+0000 7f850c53d700 1 --2- 172.21.15.6:0/3839951976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8504100c10 0x7f8504101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:06:14.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.538+0000 7f850c53d700 1 --2- 172.21.15.6:0/3839951976 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8504101e10 0x7f8504102270 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:06:14.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.538+0000 7f850c53d700 1 -- 172.21.15.6:0/3839951976 >> 172.21.15.6:0/3839951976 conn(0x7f85040fc1d0 msgr2=0x7f85040fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:06:14.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.538+0000 7f850c53d700 1 -- 172.21.15.6:0/3839951976 shutdown_connections 2024-03-24T12:06:14.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.538+0000 7f850c53d700 1 -- 172.21.15.6:0/3839951976 wait complete. 2024-03-24T12:06:14.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.539+0000 7f850c53d700 1 Processor -- start 2024-03-24T12:06:14.540 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.539+0000 7f850c53d700 1 -- start start 2024-03-24T12:06:14.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.539+0000 7f850c53d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8504100c10 0x7f850406aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:06:14.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.539+0000 7f850c53d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8504101e10 0x7f850419bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:06:14.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.539+0000 7f850c53d700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85041027b0 0x7f850419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:06:14.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.539+0000 7f850c53d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850419ed80 con 0x7f8504101e10 2024-03-24T12:06:14.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.539+0000 7f850c53d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850419eef0 con 0x7f8504100c10 2024-03-24T12:06:14.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.539+0000 7f850c53d700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850419f0b0 con 0x7f85041027b0 2024-03-24T12:06:14.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.539+0000 7f850aada700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85041027b0 0x7f850419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:06:14.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.539+0000 7f8509ad8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8504101e10 0x7f850419bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:06:14.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.539+0000 7f850a2d9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8504100c10 0x7f850406aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:06:14.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.539+0000 7f850a2d9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8504100c10 0x7f850406aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:55026/0 (socket says 172.21.15.6:55026) 2024-03-24T12:06:14.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.539+0000 7f8509ad8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8504101e10 0x7f850419bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42816/0 (socket says 172.21.15.6:42816) 2024-03-24T12:06:14.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.539+0000 7f850aada700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85041027b0 0x7f850419e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:49052/0 (socket says 172.21.15.6:49052) 2024-03-24T12:06:14.541 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.539+0000 7f850a2d9700 1 -- 172.21.15.6:0/4234130447 learned_addr learned my addr 172.21.15.6:0/4234130447 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:06:14.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.540+0000 7f850a2d9700 1 -- 172.21.15.6:0/4234130447 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85041027b0 msgr2=0x7f850419e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:06:14.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.540+0000 7f850a2d9700 1 --2- 172.21.15.6:0/4234130447 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85041027b0 0x7f850419e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:06:14.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.540+0000 7f850a2d9700 1 -- 172.21.15.6:0/4234130447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8504101e10 msgr2=0x7f850419bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:06:14.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.540+0000 7f850a2d9700 1 --2- 172.21.15.6:0/4234130447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8504101e10 0x7f850419bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:06:14.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.540+0000 7f850a2d9700 1 -- 172.21.15.6:0/4234130447 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f40095f0 con 0x7f8504100c10 2024-03-24T12:06:14.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.540+0000 7f8509ad8700 1 --2- 172.21.15.6:0/4234130447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8504101e10 0x7f850419bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:06:14.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.540+0000 7f850a2d9700 1 --2- 172.21.15.6:0/4234130447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8504100c10 0x7f850406aed0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f84fc007ee0 tx=0x7f84fc00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:06:14.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.540+0000 7f85037fe700 1 -- 172.21.15.6:0/4234130447 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc00dd20 con 0x7f8504100c10 2024-03-24T12:06:14.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.540+0000 7f850c53d700 1 -- 172.21.15.6:0/4234130447 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f8009520 con 0x7f8504100c10 2024-03-24T12:06:14.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.540+0000 7f85037fe700 1 -- 172.21.15.6:0/4234130447 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f84fc00f040 con 0x7f8504100c10 2024-03-24T12:06:14.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.540+0000 7f85037fe700 1 -- 172.21.15.6:0/4234130447 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc014600 con 0x7f8504100c10 2024-03-24T12:06:14.542 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.540+0000 7f850c53d700 1 -- 172.21.15.6:0/4234130447 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8504196190 con 0x7f8504100c10 2024-03-24T12:06:14.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.541+0000 7f850c53d700 1 -- 172.21.15.6:0/4234130447 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85041077f0 con 0x7f8504100c10 2024-03-24T12:06:14.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.541+0000 7f85037fe700 1 -- 172.21.15.6:0/4234130447 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 14) v1 ==== 91023+0+0 (secure 0 0 0) 0x7f84fc015400 con 0x7f8504100c10 2024-03-24T12:06:14.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.542+0000 7f85037fe700 1 --2- 172.21.15.6:0/4234130447 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f84ec06d4e0 0x7f84ec06f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:06:14.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.542+0000 7f85037fe700 1 -- 172.21.15.6:0/4234130447 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f84fc08b620 con 0x7f8504100c10 2024-03-24T12:06:14.543 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.542+0000 7f8509ad8700 1 --2- 172.21.15.6:0/4234130447 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f84ec06d4e0 0x7f84ec06f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:06:14.544 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.542+0000 7f8509ad8700 1 --2- 172.21.15.6:0/4234130447 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f84ec06d4e0 0x7f84ec06f9a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009490 tx=0x7f84f801d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:06:14.547 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.547+0000 7f85037fe700 1 -- 172.21.15.6:0/4234130447 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f84fc0563d0 con 0x7f8504100c10 2024-03-24T12:06:14.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:14.765+0000 7f850c53d700 1 -- 172.21.15.6:0/4234130447 --> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f850410bc00 con 0x7f84ec06d4e0 2024-03-24T12:06:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:14 smithi090 ceph-mon[33646]: Detected new or changed devices on smithi006 2024-03-24T12:06:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:14 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:14 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:14 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:06:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:14 smithi090 ceph-mon[33646]: Adjusting osd_memory_target on smithi006 to 16069M 2024-03-24T12:06:15.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:14 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:15.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:14 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:15.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:14 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:06:15.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:14 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:15.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:14 smithi090 ceph-mon[33646]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:15.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:14 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:06:15.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:14 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:15.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:14 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:06:15.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:14 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[41734]: Detected new or changed devices on smithi006 2024-03-24T12:06:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:06:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[41734]: Adjusting osd_memory_target on smithi006 to 16069M 2024-03-24T12:06:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:06:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[41734]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:06:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:15.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:06:15.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:15.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[30108]: Detected new or changed devices on smithi006 2024-03-24T12:06:15.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:15.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:15.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:06:15.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[30108]: Adjusting osd_memory_target on smithi006 to 16069M 2024-03-24T12:06:15.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:15.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:15.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:06:15.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:15.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[30108]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:15.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:06:15.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:15.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:06:15.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:14 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:16.049 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:15 smithi006 ceph-mon[30108]: from='client.24142 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:06:16.049 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:15 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:06:16.049 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:15 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:06:16.049 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:15 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:16.049 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:15 smithi006 ceph-mon[41734]: from='client.24142 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:06:16.049 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:15 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:06:16.049 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:15 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:06:16.049 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:15 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:16.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:15 smithi090 ceph-mon[33646]: from='client.24142 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:06:16.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:15 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:06:16.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:15 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:06:16.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:15 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:17.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:16 smithi090 ceph-mon[33646]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:17.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:16 smithi006 ceph-mon[30108]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:17.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:16 smithi006 ceph-mon[41734]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/951136349' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b00997af-0f11-4f04-ae58-81af568256cf"}]: dispatch 2024-03-24T12:06:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b00997af-0f11-4f04-ae58-81af568256cf"}]: dispatch 2024-03-24T12:06:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b00997af-0f11-4f04-ae58-81af568256cf"}]': finished 2024-03-24T12:06:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:17 smithi090 ceph-mon[33646]: osdmap e10: 2 total, 1 up, 2 in 2024-03-24T12:06:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:17 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T12:06:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/951136349' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b00997af-0f11-4f04-ae58-81af568256cf"}]: dispatch 2024-03-24T12:06:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b00997af-0f11-4f04-ae58-81af568256cf"}]: dispatch 2024-03-24T12:06:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b00997af-0f11-4f04-ae58-81af568256cf"}]': finished 2024-03-24T12:06:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:17 smithi006 ceph-mon[30108]: osdmap e10: 2 total, 1 up, 2 in 2024-03-24T12:06:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:17 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T12:06:18.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/951136349' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b00997af-0f11-4f04-ae58-81af568256cf"}]: dispatch 2024-03-24T12:06:18.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b00997af-0f11-4f04-ae58-81af568256cf"}]: dispatch 2024-03-24T12:06:18.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b00997af-0f11-4f04-ae58-81af568256cf"}]': finished 2024-03-24T12:06:18.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:17 smithi006 ceph-mon[41734]: osdmap e10: 2 total, 1 up, 2 in 2024-03-24T12:06:18.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:17 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T12:06:19.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:18 smithi090 ceph-mon[33646]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:19.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2545085169' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:06:19.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:18 smithi006 ceph-mon[30108]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:19.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2545085169' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:06:19.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:18 smithi006 ceph-mon[41734]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:19.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2545085169' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:06:20.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:19 smithi090 ceph-mon[33646]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:20.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:19 smithi006 ceph-mon[30108]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:20.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:19 smithi006 ceph-mon[41734]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:22.070 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:21 smithi006 ceph-mon[30108]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:22.071 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:21 smithi006 ceph-mon[41734]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:22.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:21 smithi090 ceph-mon[33646]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:24.113 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:23 smithi006 ceph-mon[30108]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:24.114 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:23 smithi006 ceph-mon[41734]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:24.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:23 smithi090 ceph-mon[33646]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:25.109 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:24 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T12:06:25.109 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:24 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:25.109 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:24 smithi006 ceph-mon[41734]: Deploying daemon osd.1 on smithi006 2024-03-24T12:06:25.109 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:24 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T12:06:25.109 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:24 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:25.109 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:24 smithi006 ceph-mon[30108]: Deploying daemon osd.1 on smithi006 2024-03-24T12:06:25.329 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:24 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-24T12:06:25.329 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:24 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:25.330 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:24 smithi090 ceph-mon[33646]: Deploying daemon osd.1 on smithi006 2024-03-24T12:06:25.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:25 smithi006 ceph-mon[30108]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:25.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:25 smithi006 ceph-mon[41734]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:26.342 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:25 smithi090 ceph-mon[33646]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:28.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:27 smithi090 ceph-mon[33646]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:28.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:27 smithi006 ceph-mon[30108]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:28.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:27 smithi006 ceph-mon[41734]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:30.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:30 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:30.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:30 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:06:30.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:30 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:30.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:30 smithi006 ceph-mon[30108]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:30.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:30 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:30.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:30 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:06:30.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:30 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:30.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:30 smithi006 ceph-mon[41734]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:30 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:30 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:06:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:30 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:30 smithi090 ceph-mon[33646]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:32.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:31 smithi090 ceph-mon[33646]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:32.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:31 smithi006 ceph-mon[30108]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:32.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:31 smithi006 ceph-mon[41734]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:32.970 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:32.970+0000 7f85037fe700 1 -- 172.21.15.6:0/4234130447 <== mgr.14152 v2:172.21.15.6:6800/3847865179 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f850410bc00 con 0x7f84ec06d4e0 2024-03-24T12:06:32.970 INFO:teuthology.orchestra.run.smithi006.stdout:Created osd(s) 1 on host 'smithi006' 2024-03-24T12:06:32.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:32.972+0000 7f850c53d700 1 -- 172.21.15.6:0/4234130447 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f84ec06d4e0 msgr2=0x7f84ec06f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:06:32.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:32.972+0000 7f850c53d700 1 --2- 172.21.15.6:0/4234130447 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f84ec06d4e0 0x7f84ec06f9a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009490 tx=0x7f84f801d040 comp rx=0 tx=0).stop 2024-03-24T12:06:32.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:32.972+0000 7f850c53d700 1 -- 172.21.15.6:0/4234130447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8504100c10 msgr2=0x7f850406aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:06:32.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:32.972+0000 7f850c53d700 1 --2- 172.21.15.6:0/4234130447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8504100c10 0x7f850406aed0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f84fc007ee0 tx=0x7f84fc00d3b0 comp rx=0 tx=0).stop 2024-03-24T12:06:32.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:32.972+0000 7f850c53d700 1 -- 172.21.15.6:0/4234130447 shutdown_connections 2024-03-24T12:06:32.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:32.972+0000 7f850c53d700 1 --2- 172.21.15.6:0/4234130447 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f84ec06d4e0 0x7f84ec06f9a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:06:32.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:32.972+0000 7f850c53d700 1 --2- 172.21.15.6:0/4234130447 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85041027b0 0x7f850419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:06:32.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:32.973+0000 7f850c53d700 1 --2- 172.21.15.6:0/4234130447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8504100c10 0x7f850406aed0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:06:32.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:32.973+0000 7f850c53d700 1 --2- 172.21.15.6:0/4234130447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8504101e10 0x7f850419bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:06:32.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:32.973+0000 7f850c53d700 1 -- 172.21.15.6:0/4234130447 >> 172.21.15.6:0/4234130447 conn(0x7f85040fc1d0 msgr2=0x7f85040ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:06:32.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:32.973+0000 7f850c53d700 1 -- 172.21.15.6:0/4234130447 shutdown_connections 2024-03-24T12:06:32.973 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:32.973+0000 7f850c53d700 1 -- 172.21.15.6:0/4234130447 wait complete. 2024-03-24T12:06:33.501 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:33 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:33.501 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:33 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:33.501 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:33 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:33.501 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:33 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:06:33.502 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:33 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:33.502 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:33 smithi006 ceph-mon[30108]: from='osd.1 [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T12:06:33.502 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:33 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:33.502 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:33 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:33.502 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:33 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:06:33.502 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:33 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:33.502 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:33 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:33.502 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:33 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:33.502 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:33 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:06:33.502 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:33 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:33.502 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:33 smithi006 ceph-mon[41734]: from='osd.1 [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T12:06:33.503 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:33 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:33.503 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:33 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:33.503 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:33 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:06:33.529 DEBUG:teuthology.orchestra.run.smithi006:osd.1> sudo journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.1.service 2024-03-24T12:06:33.531 INFO:tasks.cephadm:Deploying osd.2 on smithi006 with /dev/vg_nvme/lv_2... 2024-03-24T12:06:33.531 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-24T12:06:33.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:33 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:33.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:33 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:33.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:33 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:33.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:33 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:06:33.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:33 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:33.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:33 smithi090 ceph-mon[33646]: from='osd.1 [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-24T12:06:33.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:33 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:33.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:33 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:33.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:33 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:06:34.229 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:06:34.591 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:34 smithi006 ceph-mon[41734]: from='osd.1 [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-24T12:06:34.591 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:34 smithi006 ceph-mon[41734]: osdmap e11: 2 total, 1 up, 2 in 2024-03-24T12:06:34.592 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:34 smithi006 ceph-mon[41734]: from='osd.1 [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-24T12:06:34.592 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:34 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T12:06:34.592 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:34 smithi006 ceph-mon[41734]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:34.592 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:34 smithi006 ceph-mon[30108]: from='osd.1 [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-24T12:06:34.592 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:34 smithi006 ceph-mon[30108]: osdmap e11: 2 total, 1 up, 2 in 2024-03-24T12:06:34.592 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:34 smithi006 ceph-mon[30108]: from='osd.1 [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-24T12:06:34.592 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:34 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T12:06:34.592 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:34 smithi006 ceph-mon[30108]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:34.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:34 smithi090 ceph-mon[33646]: from='osd.1 [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-24T12:06:34.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:34 smithi090 ceph-mon[33646]: osdmap e11: 2 total, 1 up, 2 in 2024-03-24T12:06:34.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:34 smithi090 ceph-mon[33646]: from='osd.1 [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-24T12:06:34.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:34 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T12:06:34.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:34 smithi090 ceph-mon[33646]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:34.887 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:06:34 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-1[52194]: 2024-03-24T12:06:34.716+0000 7fa94b84d700 -1 osd.1 0 waiting for initial osdmap 2024-03-24T12:06:34.887 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:06:34 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-1[52194]: 2024-03-24T12:06:34.719+0000 7fa943638700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T12:06:35.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:35 smithi006 ceph-mon[30108]: from='osd.1 [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-03-24T12:06:35.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:35 smithi006 ceph-mon[30108]: osdmap e12: 2 total, 1 up, 2 in 2024-03-24T12:06:35.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:35 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T12:06:35.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:35 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T12:06:35.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:35 smithi006 ceph-mon[30108]: from='osd.1 [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601]' entity='osd.1' 2024-03-24T12:06:35.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:35 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T12:06:35.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:35 smithi006 ceph-mon[41734]: from='osd.1 [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-03-24T12:06:35.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:35 smithi006 ceph-mon[41734]: osdmap e12: 2 total, 1 up, 2 in 2024-03-24T12:06:35.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:35 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T12:06:35.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:35 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T12:06:35.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:35 smithi006 ceph-mon[41734]: from='osd.1 [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601]' entity='osd.1' 2024-03-24T12:06:35.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:35 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T12:06:35.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:35 smithi090 ceph-mon[33646]: from='osd.1 [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-03-24T12:06:35.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:35 smithi090 ceph-mon[33646]: osdmap e12: 2 total, 1 up, 2 in 2024-03-24T12:06:35.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:35 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T12:06:35.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:35 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T12:06:35.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:35 smithi090 ceph-mon[33646]: from='osd.1 [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601]' entity='osd.1' 2024-03-24T12:06:35.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:35 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T12:06:36.415 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:36 smithi006 ceph-mon[41734]: purged_snaps scrub starts 2024-03-24T12:06:36.415 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:36 smithi006 ceph-mon[41734]: purged_snaps scrub ok 2024-03-24T12:06:36.415 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:36 smithi006 ceph-mon[41734]: osd.1 [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601] boot 2024-03-24T12:06:36.416 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:36 smithi006 ceph-mon[41734]: osdmap e13: 2 total, 2 up, 2 in 2024-03-24T12:06:36.416 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:36 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T12:06:36.416 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:36 smithi006 ceph-mon[41734]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:36.416 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:36 smithi006 ceph-mon[30108]: purged_snaps scrub starts 2024-03-24T12:06:36.416 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:36 smithi006 ceph-mon[30108]: purged_snaps scrub ok 2024-03-24T12:06:36.416 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:36 smithi006 ceph-mon[30108]: osd.1 [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601] boot 2024-03-24T12:06:36.416 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:36 smithi006 ceph-mon[30108]: osdmap e13: 2 total, 2 up, 2 in 2024-03-24T12:06:36.416 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:36 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T12:06:36.416 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:36 smithi006 ceph-mon[30108]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:36.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:36 smithi090 ceph-mon[33646]: purged_snaps scrub starts 2024-03-24T12:06:36.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:36 smithi090 ceph-mon[33646]: purged_snaps scrub ok 2024-03-24T12:06:36.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:36 smithi090 ceph-mon[33646]: osd.1 [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601] boot 2024-03-24T12:06:36.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:36 smithi090 ceph-mon[33646]: osdmap e13: 2 total, 2 up, 2 in 2024-03-24T12:06:36.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:36 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T12:06:36.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:36 smithi090 ceph-mon[33646]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-24T12:06:37.558 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-24T12:06:37.559 INFO:teuthology.orchestra.run.smithi006.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-24T12:06:37.559 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10+0 records in 2024-03-24T12:06:37.559 INFO:teuthology.orchestra.run.smithi006.stderr:10+0 records out 2024-03-24T12:06:37.559 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123654 s, 848 MB/s 2024-03-24T12:06:37.559 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping successful for: 2024-03-24T12:06:38.205 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[30108]: Detected new or changed devices on smithi006 2024-03-24T12:06:38.205 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:38.206 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:38.206 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:06:38.206 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:06:38.206 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[30108]: Adjusting osd_memory_target on smithi006 to 8034M 2024-03-24T12:06:38.206 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:38.206 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:38.206 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:06:38.206 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:38.206 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[30108]: osdmap e14: 2 total, 2 up, 2 in 2024-03-24T12:06:38.206 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[30108]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:38.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[41734]: Detected new or changed devices on smithi006 2024-03-24T12:06:38.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:38.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:38.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:06:38.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:06:38.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[41734]: Adjusting osd_memory_target on smithi006 to 8034M 2024-03-24T12:06:38.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:38.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:38.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:06:38.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:38.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[41734]: osdmap e14: 2 total, 2 up, 2 in 2024-03-24T12:06:38.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:38 smithi006 ceph-mon[41734]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:38.208 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi006:vg_nvme/lv_2 2024-03-24T12:06:38.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:38 smithi090 ceph-mon[33646]: Detected new or changed devices on smithi006 2024-03-24T12:06:38.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:38 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:38.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:38 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:38.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:38 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:06:38.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:38 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:06:38.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:38 smithi090 ceph-mon[33646]: Adjusting osd_memory_target on smithi006 to 8034M 2024-03-24T12:06:38.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:38 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:38.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:38 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:38.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:38 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:06:38.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:38 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:38.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:38 smithi090 ceph-mon[33646]: osdmap e14: 2 total, 2 up, 2 in 2024-03-24T12:06:38.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:38 smithi090 ceph-mon[33646]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:38.450 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:06:39.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.941+0000 7fa4921f6700 1 -- 172.21.15.6:0/4294165595 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa48c102740 msgr2=0x7fa48c108ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:06:39.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.941+0000 7fa4921f6700 1 --2- 172.21.15.6:0/4294165595 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa48c102740 0x7fa48c108ff0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa48400b0d0 tx=0x7fa48400b3e0 comp rx=0 tx=0).stop 2024-03-24T12:06:39.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.941+0000 7fa4921f6700 1 -- 172.21.15.6:0/4294165595 shutdown_connections 2024-03-24T12:06:39.944 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.941+0000 7fa4921f6700 1 --2- 172.21.15.6:0/4294165595 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa48c102740 0x7fa48c108ff0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:06:39.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.941+0000 7fa4921f6700 1 --2- 172.21.15.6:0/4294165595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa48c100ba0 0x7fa48c100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:06:39.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.941+0000 7fa4921f6700 1 --2- 172.21.15.6:0/4294165595 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa48c101da0 0x7fa48c102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:06:39.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.941+0000 7fa4921f6700 1 -- 172.21.15.6:0/4294165595 >> 172.21.15.6:0/4294165595 conn(0x7fa48c0fc140 msgr2=0x7fa48c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:06:39.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.941+0000 7fa4921f6700 1 -- 172.21.15.6:0/4294165595 shutdown_connections 2024-03-24T12:06:39.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.941+0000 7fa4921f6700 1 -- 172.21.15.6:0/4294165595 wait complete. 2024-03-24T12:06:39.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.942+0000 7fa4921f6700 1 Processor -- start 2024-03-24T12:06:39.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.942+0000 7fa4921f6700 1 -- start start 2024-03-24T12:06:39.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.942+0000 7fa4921f6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa48c100ba0 0x7fa48c1994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:06:39.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.942+0000 7fa4921f6700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa48c101da0 0x7fa48c199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:06:39.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.942+0000 7fa4921f6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa48c102740 0x7fa48c1a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:06:39.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.942+0000 7fa4921f6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c1a1060 con 0x7fa48c102740 2024-03-24T12:06:39.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.942+0000 7fa4921f6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c1a11a0 con 0x7fa48c100ba0 2024-03-24T12:06:39.945 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.942+0000 7fa4921f6700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c1a1330 con 0x7fa48c101da0 2024-03-24T12:06:39.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.942+0000 7fa48affd700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa48c101da0 0x7fa48c199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:06:39.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.942+0000 7fa48bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa48c102740 0x7fa48c1a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:06:39.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.942+0000 7fa48affd700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa48c101da0 0x7fa48c199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:35092/0 (socket says 172.21.15.6:35092) 2024-03-24T12:06:39.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.942+0000 7fa48bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa48c102740 0x7fa48c1a0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59068/0 (socket says 172.21.15.6:59068) 2024-03-24T12:06:39.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.942+0000 7fa48affd700 1 -- 172.21.15.6:0/4083521125 learned_addr learned my addr 172.21.15.6:0/4083521125 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:06:39.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.943+0000 7fa48b7fe700 1 --2- 172.21.15.6:0/4083521125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa48c100ba0 0x7fa48c1994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:06:39.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.943+0000 7fa48affd700 1 -- 172.21.15.6:0/4083521125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa48c100ba0 msgr2=0x7fa48c1994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:06:39.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.943+0000 7fa48affd700 1 --2- 172.21.15.6:0/4083521125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa48c100ba0 0x7fa48c1994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:06:39.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.943+0000 7fa48affd700 1 -- 172.21.15.6:0/4083521125 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa48c102740 msgr2=0x7fa48c1a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:06:39.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.943+0000 7fa48affd700 1 --2- 172.21.15.6:0/4083521125 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa48c102740 0x7fa48c1a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:06:39.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.943+0000 7fa48affd700 1 -- 172.21.15.6:0/4083521125 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa484009d00 con 0x7fa48c101da0 2024-03-24T12:06:39.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.943+0000 7fa48bfff700 1 --2- 172.21.15.6:0/4083521125 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa48c102740 0x7fa48c1a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:06:39.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.943+0000 7fa48b7fe700 1 --2- 172.21.15.6:0/4083521125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa48c100ba0 0x7fa48c1994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T12:06:39.946 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.943+0000 7fa48affd700 1 --2- 172.21.15.6:0/4083521125 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa48c101da0 0x7fa48c199a10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa48000e9d0 tx=0x7fa48000ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:06:39.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.943+0000 7fa488ff9700 1 -- 172.21.15.6:0/4083521125 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa480009790 con 0x7fa48c101da0 2024-03-24T12:06:39.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.943+0000 7fa488ff9700 1 -- 172.21.15.6:0/4083521125 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fa48000cbd0 con 0x7fa48c101da0 2024-03-24T12:06:39.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.943+0000 7fa4921f6700 1 -- 172.21.15.6:0/4083521125 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa48c1a1610 con 0x7fa48c101da0 2024-03-24T12:06:39.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.943+0000 7fa488ff9700 1 -- 172.21.15.6:0/4083521125 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa480018420 con 0x7fa48c101da0 2024-03-24T12:06:39.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.943+0000 7fa4921f6700 1 -- 172.21.15.6:0/4083521125 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48c1a1b10 con 0x7fa48c101da0 2024-03-24T12:06:39.947 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.944+0000 7fa4921f6700 1 -- 172.21.15.6:0/4083521125 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa48c106770 con 0x7fa48c101da0 2024-03-24T12:06:39.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.944+0000 7fa488ff9700 1 -- 172.21.15.6:0/4083521125 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 14) v1 ==== 91023+0+0 (secure 0 0 0) 0x7fa480010430 con 0x7fa48c101da0 2024-03-24T12:06:39.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.945+0000 7fa488ff9700 1 --2- 172.21.15.6:0/4083521125 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fa47406d4e0 0x7fa47406f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:06:39.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.945+0000 7fa488ff9700 1 -- 172.21.15.6:0/4083521125 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2199+0+0 (secure 0 0 0) 0x7fa480014070 con 0x7fa48c101da0 2024-03-24T12:06:39.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.945+0000 7fa48b7fe700 1 --2- 172.21.15.6:0/4083521125 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fa47406d4e0 0x7fa47406f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:06:39.948 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.945+0000 7fa48b7fe700 1 --2- 172.21.15.6:0/4083521125 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fa47406d4e0 0x7fa47406f9a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa47c005fd0 tx=0x7fa47c005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:06:39.950 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:39.950+0000 7fa488ff9700 1 -- 172.21.15.6:0/4083521125 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa480052d20 con 0x7fa48c101da0 2024-03-24T12:06:40.161 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:40.160+0000 7fa4921f6700 1 -- 172.21.15.6:0/4083521125 --> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa48c0620b0 con 0x7fa47406d4e0 2024-03-24T12:06:40.161 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:39 smithi006 ceph-mon[30108]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:40.161 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:39 smithi006 ceph-mon[41734]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:40.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:39 smithi090 ceph-mon[33646]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:41.026 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:40 smithi006 ceph-mon[30108]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:06:41.026 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:40 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:06:41.026 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:40 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:06:41.026 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:40 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:41.026 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:40 smithi006 ceph-mon[41734]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:06:41.026 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:40 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:06:41.027 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:40 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:06:41.027 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:40 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:41.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:40 smithi090 ceph-mon[33646]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:06:41.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:40 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:06:41.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:40 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:06:41.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:40 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:41.978 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:41 smithi006 ceph-mon[30108]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:41.979 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:41 smithi006 ceph-mon[41734]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:42.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:41 smithi090 ceph-mon[33646]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:43.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:42 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1187984956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eec34136-82f9-41eb-bbac-f0d6edf7e73e"}]: dispatch 2024-03-24T12:06:43.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:42 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1187984956' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eec34136-82f9-41eb-bbac-f0d6edf7e73e"}]': finished 2024-03-24T12:06:43.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:42 smithi006 ceph-mon[30108]: osdmap e15: 3 total, 2 up, 3 in 2024-03-24T12:06:43.198 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:42 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T12:06:43.198 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:42 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1187984956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eec34136-82f9-41eb-bbac-f0d6edf7e73e"}]: dispatch 2024-03-24T12:06:43.198 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:42 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1187984956' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eec34136-82f9-41eb-bbac-f0d6edf7e73e"}]': finished 2024-03-24T12:06:43.199 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:42 smithi006 ceph-mon[41734]: osdmap e15: 3 total, 2 up, 3 in 2024-03-24T12:06:43.199 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:42 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T12:06:43.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:42 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1187984956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eec34136-82f9-41eb-bbac-f0d6edf7e73e"}]: dispatch 2024-03-24T12:06:43.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:42 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1187984956' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eec34136-82f9-41eb-bbac-f0d6edf7e73e"}]': finished 2024-03-24T12:06:43.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:42 smithi090 ceph-mon[33646]: osdmap e15: 3 total, 2 up, 3 in 2024-03-24T12:06:43.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:42 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T12:06:44.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:43 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/131843844' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:06:44.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:43 smithi006 ceph-mon[30108]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:44.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:43 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/131843844' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:06:44.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:43 smithi006 ceph-mon[41734]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:43 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/131843844' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:06:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:43 smithi090 ceph-mon[33646]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:46.064 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:45 smithi006 ceph-mon[30108]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:46.064 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:45 smithi006 ceph-mon[41734]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:46.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:45 smithi090 ceph-mon[33646]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:47 smithi006 ceph-mon[30108]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:48.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:47 smithi006 ceph-mon[41734]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:48.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:47 smithi090 ceph-mon[33646]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:49.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:49 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T12:06:49.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:49 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:49.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:49 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T12:06:49.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:49 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:49.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:49 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-24T12:06:49.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:49 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:06:50.658 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:50 smithi006 ceph-mon[30108]: Deploying daemon osd.2 on smithi006 2024-03-24T12:06:50.658 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:50 smithi006 ceph-mon[30108]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:50.659 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:50 smithi006 ceph-mon[41734]: Deploying daemon osd.2 on smithi006 2024-03-24T12:06:50.659 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:50 smithi006 ceph-mon[41734]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:50.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:50 smithi090 ceph-mon[33646]: Deploying daemon osd.2 on smithi006 2024-03-24T12:06:50.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:50 smithi090 ceph-mon[33646]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:52.148 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:51 smithi006 ceph-mon[30108]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:52.149 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:51 smithi006 ceph-mon[41734]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:52.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:51 smithi090 ceph-mon[33646]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:54.124 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:53 smithi006 ceph-mon[30108]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:54.124 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:53 smithi006 ceph-mon[41734]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:54.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:53 smithi090 ceph-mon[33646]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:55.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:55 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:55.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:55 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:55.722 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:55 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:06:55.722 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:55 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:55.722 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:55 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:55.723 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:55 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:06:56.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:55 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:56.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:55 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:56.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:55 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:06:56.834 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:56 smithi006 ceph-mon[30108]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:56.835 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:56 smithi006 ceph-mon[41734]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:57.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:56 smithi090 ceph-mon[33646]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:58.363 INFO:teuthology.orchestra.run.smithi006.stdout:Created osd(s) 2 on host 'smithi006' 2024-03-24T12:06:58.364 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:58.363+0000 7fa488ff9700 1 -- 172.21.15.6:0/4083521125 <== mgr.14152 v2:172.21.15.6:6800/3847865179 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa48c0620b0 con 0x7fa47406d4e0 2024-03-24T12:06:58.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:58.366+0000 7fa4921f6700 1 -- 172.21.15.6:0/4083521125 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fa47406d4e0 msgr2=0x7fa47406f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:06:58.365 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:58.366+0000 7fa4921f6700 1 --2- 172.21.15.6:0/4083521125 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fa47406d4e0 0x7fa47406f9a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa47c005fd0 tx=0x7fa47c005f00 comp rx=0 tx=0).stop 2024-03-24T12:06:58.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:58.366+0000 7fa4921f6700 1 -- 172.21.15.6:0/4083521125 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa48c101da0 msgr2=0x7fa48c199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:06:58.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:58.366+0000 7fa4921f6700 1 --2- 172.21.15.6:0/4083521125 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa48c101da0 0x7fa48c199a10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa48000e9d0 tx=0x7fa48000ece0 comp rx=0 tx=0).stop 2024-03-24T12:06:58.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:58.366+0000 7fa4921f6700 1 -- 172.21.15.6:0/4083521125 shutdown_connections 2024-03-24T12:06:58.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:58.366+0000 7fa4921f6700 1 --2- 172.21.15.6:0/4083521125 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fa47406d4e0 0x7fa47406f9a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:06:58.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:58.366+0000 7fa4921f6700 1 --2- 172.21.15.6:0/4083521125 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa48c102740 0x7fa48c1a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:06:58.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:58.366+0000 7fa4921f6700 1 --2- 172.21.15.6:0/4083521125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa48c100ba0 0x7fa48c1994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:06:58.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:58.366+0000 7fa4921f6700 1 --2- 172.21.15.6:0/4083521125 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa48c101da0 0x7fa48c199a10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:06:58.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:58.366+0000 7fa4921f6700 1 -- 172.21.15.6:0/4083521125 >> 172.21.15.6:0/4083521125 conn(0x7fa48c0fc140 msgr2=0x7fa48c06a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:06:58.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:58.366+0000 7fa4921f6700 1 -- 172.21.15.6:0/4083521125 shutdown_connections 2024-03-24T12:06:58.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:06:58.366+0000 7fa4921f6700 1 -- 172.21.15.6:0/4083521125 wait complete. 2024-03-24T12:06:58.690 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:58 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:58.691 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:58 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:58.691 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:58 smithi006 ceph-mon[30108]: from='osd.2 [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T12:06:58.691 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:58 smithi006 ceph-mon[30108]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:58.691 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:58 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:58.691 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:58 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:58.691 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:58 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:58.691 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:58 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:58.691 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:58 smithi006 ceph-mon[41734]: from='osd.2 [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T12:06:58.691 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:58 smithi006 ceph-mon[41734]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:58.691 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:58 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:58.691 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:58 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:58.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:58 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:58.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:58 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:58.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:58 smithi090 ceph-mon[33646]: from='osd.2 [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-24T12:06:58.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:58 smithi090 ceph-mon[33646]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:06:58.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:58 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:58.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:58 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:06:58.963 DEBUG:teuthology.orchestra.run.smithi006:osd.2> sudo journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.2.service 2024-03-24T12:06:58.965 INFO:tasks.cephadm:Deploying osd.3 on smithi006 with /dev/vg_nvme/lv_1... 2024-03-24T12:06:58.965 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-24T12:06:59.712 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:06:59.865 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:59 smithi006 ceph-mon[41734]: from='osd.2 [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-24T12:06:59.865 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:59 smithi006 ceph-mon[41734]: osdmap e16: 3 total, 2 up, 3 in 2024-03-24T12:06:59.865 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:59 smithi006 ceph-mon[41734]: from='osd.2 [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-24T12:06:59.865 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:06:59 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T12:06:59.865 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:59 smithi006 ceph-mon[30108]: from='osd.2 [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-24T12:06:59.865 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:59 smithi006 ceph-mon[30108]: osdmap e16: 3 total, 2 up, 3 in 2024-03-24T12:06:59.866 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:59 smithi006 ceph-mon[30108]: from='osd.2 [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-24T12:06:59.866 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:06:59 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T12:06:59.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:59 smithi090 ceph-mon[33646]: from='osd.2 [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-24T12:06:59.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:59 smithi090 ceph-mon[33646]: osdmap e16: 3 total, 2 up, 3 in 2024-03-24T12:06:59.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:59 smithi090 ceph-mon[33646]: from='osd.2 [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-24T12:06:59.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:06:59 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T12:07:00.139 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:06:59 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-2[58188]: 2024-03-24T12:06:59.978+0000 7f5fa1176700 -1 osd.2 0 waiting for initial osdmap 2024-03-24T12:07:00.140 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:06:59 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-2[58188]: 2024-03-24T12:06:59.981+0000 7f5f98f61700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T12:07:00.704 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:00 smithi006 ceph-mon[30108]: from='osd.2 [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-03-24T12:07:00.704 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:00 smithi006 ceph-mon[30108]: osdmap e17: 3 total, 2 up, 3 in 2024-03-24T12:07:00.704 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:00 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T12:07:00.705 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:00 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T12:07:00.705 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:00 smithi006 ceph-mon[30108]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:07:00.705 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:00 smithi006 ceph-mon[30108]: from='osd.2 [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154]' entity='osd.2' 2024-03-24T12:07:00.705 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:00 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T12:07:00.705 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:00 smithi006 ceph-mon[41734]: from='osd.2 [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-03-24T12:07:00.705 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:00 smithi006 ceph-mon[41734]: osdmap e17: 3 total, 2 up, 3 in 2024-03-24T12:07:00.705 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:00 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T12:07:00.705 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:00 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T12:07:00.705 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:00 smithi006 ceph-mon[41734]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:07:00.705 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:00 smithi006 ceph-mon[41734]: from='osd.2 [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154]' entity='osd.2' 2024-03-24T12:07:00.705 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:00 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T12:07:00.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:00 smithi090 ceph-mon[33646]: from='osd.2 [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-03-24T12:07:00.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:00 smithi090 ceph-mon[33646]: osdmap e17: 3 total, 2 up, 3 in 2024-03-24T12:07:00.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:00 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T12:07:00.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:00 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T12:07:00.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:00 smithi090 ceph-mon[33646]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-24T12:07:00.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:00 smithi090 ceph-mon[33646]: from='osd.2 [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154]' entity='osd.2' 2024-03-24T12:07:00.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:00 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T12:07:01.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:01 smithi006 ceph-mon[30108]: purged_snaps scrub starts 2024-03-24T12:07:01.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:01 smithi006 ceph-mon[30108]: purged_snaps scrub ok 2024-03-24T12:07:01.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:01 smithi006 ceph-mon[30108]: osd.2 [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154] boot 2024-03-24T12:07:01.863 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:01 smithi006 ceph-mon[30108]: osdmap e18: 3 total, 3 up, 3 in 2024-03-24T12:07:01.863 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:01 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T12:07:01.863 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:01 smithi006 ceph-mon[41734]: purged_snaps scrub starts 2024-03-24T12:07:01.863 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:01 smithi006 ceph-mon[41734]: purged_snaps scrub ok 2024-03-24T12:07:01.863 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:01 smithi006 ceph-mon[41734]: osd.2 [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154] boot 2024-03-24T12:07:01.863 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:01 smithi006 ceph-mon[41734]: osdmap e18: 3 total, 3 up, 3 in 2024-03-24T12:07:01.863 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:01 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T12:07:01.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:01 smithi090 ceph-mon[33646]: purged_snaps scrub starts 2024-03-24T12:07:01.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:01 smithi090 ceph-mon[33646]: purged_snaps scrub ok 2024-03-24T12:07:01.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:01 smithi090 ceph-mon[33646]: osd.2 [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154] boot 2024-03-24T12:07:01.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:01 smithi090 ceph-mon[33646]: osdmap e18: 3 total, 3 up, 3 in 2024-03-24T12:07:01.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:01 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T12:07:02.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:02 smithi090 ceph-mon[33646]: pgmap v76: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:02.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:02 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:07:02.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:02 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:02.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:02 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:02.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:02 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:02.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:02 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:02.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:02 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:02 smithi006 ceph-mon[30108]: pgmap v76: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:02 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:07:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:02 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:02 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:02 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:02 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:02 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:02.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:02 smithi006 ceph-mon[41734]: pgmap v76: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:02.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:02 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:07:02.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:02 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:02.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:02 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:02.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:02 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:02.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:02 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:02.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:02 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:03.112 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-24T12:07:03.113 INFO:teuthology.orchestra.run.smithi006.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-24T12:07:03.113 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10+0 records in 2024-03-24T12:07:03.113 INFO:teuthology.orchestra.run.smithi006.stderr:10+0 records out 2024-03-24T12:07:03.113 INFO:teuthology.orchestra.run.smithi006.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123838 s, 847 MB/s 2024-03-24T12:07:03.113 INFO:teuthology.orchestra.run.smithi006.stderr:--> Zapping successful for: 2024-03-24T12:07:03.682 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[30108]: Detected new or changed devices on smithi006 2024-03-24T12:07:03.683 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[30108]: Adjusting osd_memory_target on smithi006 to 5356M 2024-03-24T12:07:03.683 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:03.683 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:03.683 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-24T12:07:03.683 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[30108]: osdmap e19: 3 total, 3 up, 3 in 2024-03-24T12:07:03.683 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:07:03.683 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:07:03.683 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:03.683 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:07:03.683 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:03.683 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:07:03.683 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:03.684 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[41734]: Detected new or changed devices on smithi006 2024-03-24T12:07:03.684 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[41734]: Adjusting osd_memory_target on smithi006 to 5356M 2024-03-24T12:07:03.684 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:03.684 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:03.684 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-24T12:07:03.684 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[41734]: osdmap e19: 3 total, 3 up, 3 in 2024-03-24T12:07:03.684 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:07:03.684 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:07:03.684 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:03.684 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:07:03.684 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:03.684 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:07:03.684 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:03 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:03.710 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi006:vg_nvme/lv_1 2024-03-24T12:07:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:03 smithi090 ceph-mon[33646]: Detected new or changed devices on smithi006 2024-03-24T12:07:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:03 smithi090 ceph-mon[33646]: Adjusting osd_memory_target on smithi006 to 5356M 2024-03-24T12:07:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:03 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:03 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:03.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:03 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-24T12:07:03.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:03 smithi090 ceph-mon[33646]: osdmap e19: 3 total, 3 up, 3 in 2024-03-24T12:07:03.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:03 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:07:03.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:03 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:07:03.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:03 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:03.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:03 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:07:03.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:03 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:03.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:03 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:07:03.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:03 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:03.977 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:07:04.351 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:07:04 smithi006 sudo[62323]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T12:07:04.352 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:07:04 smithi006 sudo[62323]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T12:07:04.352 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:07:04 smithi006 sudo[62323]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T12:07:04.352 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:07:04 smithi006 sudo[62323]: pam_unix(sudo:session): session closed for user root 2024-03-24T12:07:04.614 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:07:04 smithi006 sudo[62352]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T12:07:04.614 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:07:04 smithi006 sudo[62352]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T12:07:04.614 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:07:04 smithi006 sudo[62352]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T12:07:04.614 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:07:04 smithi006 sudo[62352]: pam_unix(sudo:session): session closed for user root 2024-03-24T12:07:04.914 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:04 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-24T12:07:04.915 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:04 smithi006 ceph-mon[30108]: osdmap e20: 3 total, 3 up, 3 in 2024-03-24T12:07:04.915 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:04 smithi006 ceph-mon[30108]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:04.915 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:04 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:04.915 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:04 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-24T12:07:04.915 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:04 smithi006 ceph-mon[41734]: osdmap e20: 3 total, 3 up, 3 in 2024-03-24T12:07:04.915 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:04 smithi006 ceph-mon[41734]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:04.915 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:04 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:04.915 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:07:04 smithi006 sudo[62367]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T12:07:04.916 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:07:04 smithi006 sudo[62367]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T12:07:04.916 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:07:04 smithi006 sudo[62367]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T12:07:04.916 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:07:04 smithi006 sudo[62367]: pam_unix(sudo:session): session closed for user root 2024-03-24T12:07:04.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:04 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-24T12:07:04.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:04 smithi090 ceph-mon[33646]: osdmap e20: 3 total, 3 up, 3 in 2024-03-24T12:07:04.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:04 smithi090 ceph-mon[33646]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:04.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:04 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:05.179 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:07:04 smithi006 sudo[62378]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T12:07:05.180 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:07:04 smithi006 sudo[62378]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T12:07:05.180 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:07:04 smithi006 sudo[62378]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T12:07:05.180 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:07:04 smithi006 sudo[62378]: pam_unix(sudo:session): session closed for user root 2024-03-24T12:07:05.440 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:07:05 smithi006 sudo[62394]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T12:07:05.440 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:07:05 smithi006 sudo[62394]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T12:07:05.440 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:07:05 smithi006 sudo[62394]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T12:07:05.440 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:07:05 smithi006 sudo[62394]: pam_unix(sudo:session): session closed for user root 2024-03-24T12:07:05.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.658+0000 7f5218882700 1 -- 172.21.15.6:0/1018285054 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f52140fff60 msgr2=0x7f5214100380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:07:05.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.658+0000 7f5218882700 1 --2- 172.21.15.6:0/1018285054 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f52140fff60 0x7f5214100380 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5204009990 tx=0x7f5204009ca0 comp rx=0 tx=0).stop 2024-03-24T12:07:05.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.658+0000 7f5218882700 1 -- 172.21.15.6:0/1018285054 shutdown_connections 2024-03-24T12:07:05.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.658+0000 7f5218882700 1 --2- 172.21.15.6:0/1018285054 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52141038e0 0x7f521410a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:07:05.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.658+0000 7f5218882700 1 --2- 172.21.15.6:0/1018285054 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f52140fff60 0x7f5214100380 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:07:05.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.658+0000 7f5218882700 1 --2- 172.21.15.6:0/1018285054 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5214102f40 0x7f52141033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:07:05.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.658+0000 7f5218882700 1 -- 172.21.15.6:0/1018285054 >> 172.21.15.6:0/1018285054 conn(0x7f52140fb8e0 msgr2=0x7f52140fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:07:05.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.658+0000 7f5218882700 1 -- 172.21.15.6:0/1018285054 shutdown_connections 2024-03-24T12:07:05.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.658+0000 7f5218882700 1 -- 172.21.15.6:0/1018285054 wait complete. 2024-03-24T12:07:05.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.658+0000 7f5218882700 1 Processor -- start 2024-03-24T12:07:05.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.659+0000 7f5218882700 1 -- start start 2024-03-24T12:07:05.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.659+0000 7f5218882700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52140fff60 0x7f52141993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:07:05.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.659+0000 7f5218882700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5214102f40 0x7f5214199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:07:05.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.659+0000 7f5218882700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52141038e0 0x7f52141a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:07:05.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.659+0000 7f5218882700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52141a0f00 con 0x7f52140fff60 2024-03-24T12:07:05.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.659+0000 7f5218882700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52141a1040 con 0x7f52141038e0 2024-03-24T12:07:05.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.659+0000 7f5218882700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52141a11d0 con 0x7f5214102f40 2024-03-24T12:07:05.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.659+0000 7f5211d9b700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5214102f40 0x7f5214199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:07:05.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.659+0000 7f5211d9b700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5214102f40 0x7f5214199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:39522/0 (socket says 172.21.15.6:39522) 2024-03-24T12:07:05.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.659+0000 7f5211d9b700 1 -- 172.21.15.6:0/3823433311 learned_addr learned my addr 172.21.15.6:0/3823433311 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:07:05.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.659+0000 7f521259c700 1 --2- 172.21.15.6:0/3823433311 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52140fff60 0x7f52141993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:07:05.660 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.659+0000 7f5212d9d700 1 --2- 172.21.15.6:0/3823433311 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52141038e0 0x7f52141a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:07:05.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.659+0000 7f5211d9b700 1 -- 172.21.15.6:0/3823433311 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52141038e0 msgr2=0x7f52141a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:07:05.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.659+0000 7f5211d9b700 1 --2- 172.21.15.6:0/3823433311 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52141038e0 0x7f52141a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:07:05.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.659+0000 7f5211d9b700 1 -- 172.21.15.6:0/3823433311 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52140fff60 msgr2=0x7f52141993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:07:05.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.659+0000 7f5211d9b700 1 --2- 172.21.15.6:0/3823433311 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52140fff60 0x7f52141993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:07:05.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.659+0000 7f5211d9b700 1 -- 172.21.15.6:0/3823433311 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52040095f0 con 0x7f5214102f40 2024-03-24T12:07:05.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.659+0000 7f5211d9b700 1 --2- 172.21.15.6:0/3823433311 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5214102f40 0x7f5214199920 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f520800e9d0 tx=0x7f520800ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:07:05.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.660+0000 7f52037fe700 1 -- 172.21.15.6:0/3823433311 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5208009790 con 0x7f5214102f40 2024-03-24T12:07:05.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.660+0000 7f52037fe700 1 -- 172.21.15.6:0/3823433311 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f520800cbd0 con 0x7f5214102f40 2024-03-24T12:07:05.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.660+0000 7f5218882700 1 -- 172.21.15.6:0/3823433311 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52141a14b0 con 0x7f5214102f40 2024-03-24T12:07:05.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.660+0000 7f52037fe700 1 -- 172.21.15.6:0/3823433311 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5208018420 con 0x7f5214102f40 2024-03-24T12:07:05.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.660+0000 7f5218882700 1 -- 172.21.15.6:0/3823433311 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52141a1a00 con 0x7f5214102f40 2024-03-24T12:07:05.661 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.661+0000 7f5218882700 1 -- 172.21.15.6:0/3823433311 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5214068930 con 0x7f5214102f40 2024-03-24T12:07:05.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.661+0000 7f52037fe700 1 -- 172.21.15.6:0/3823433311 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 14) v1 ==== 91023+0+0 (secure 0 0 0) 0x7f5208010430 con 0x7f5214102f40 2024-03-24T12:07:05.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.661+0000 7f52037fe700 1 --2- 172.21.15.6:0/3823433311 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f51fc06d4e0 0x7f51fc06f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:07:05.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.662+0000 7f52037fe700 1 -- 172.21.15.6:0/3823433311 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3017+0+0 (secure 0 0 0) 0x7f5208014070 con 0x7f5214102f40 2024-03-24T12:07:05.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.662+0000 7f521259c700 1 --2- 172.21.15.6:0/3823433311 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f51fc06d4e0 0x7f51fc06f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:07:05.662 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.662+0000 7f521259c700 1 --2- 172.21.15.6:0/3823433311 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f51fc06d4e0 0x7f51fc06f9a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f52040059d0 tx=0x7f5204005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:07:05.665 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.666+0000 7f52037fe700 1 -- 172.21.15.6:0/3823433311 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5208052d20 con 0x7f5214102f40 2024-03-24T12:07:05.710 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:07:05 smithi006 sudo[62424]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T12:07:05.710 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:07:05 smithi006 sudo[62424]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T12:07:05.710 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:07:05 smithi006 sudo[62424]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T12:07:05.710 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:07:05 smithi006 sudo[62424]: pam_unix(sudo:session): session closed for user root 2024-03-24T12:07:05.881 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.882+0000 7f5218882700 1 -- 172.21.15.6:0/3823433311 --> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f5214101d40 con 0x7f51fc06d4e0 2024-03-24T12:07:05.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:05 smithi006 sudo[62454]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T12:07:05.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:05 smithi006 sudo[62454]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T12:07:05.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:05 smithi006 sudo[62454]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T12:07:05.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:05 smithi006 sudo[62454]: pam_unix(sudo:session): session closed for user root 2024-03-24T12:07:05.993 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:05.994+0000 7f52037fe700 1 -- 172.21.15.6:0/3823433311 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mgrmap(e 15) v1 ==== 91079+0+0 (secure 0 0 0) 0x7f52080106e0 con 0x7f5214102f40 2024-03-24T12:07:06.244 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:05 smithi090 ceph-mon[33646]: osdmap e21: 3 total, 3 up, 3 in 2024-03-24T12:07:06.244 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:05 smithi090 ceph-mon[33646]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T12:07:06.244 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:06 smithi090 sudo[36466]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T12:07:06.244 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:06 smithi090 sudo[36466]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T12:07:06.245 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:06 smithi090 sudo[36466]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T12:07:06.245 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:06 smithi090 sudo[36466]: pam_unix(sudo:session): session closed for user root 2024-03-24T12:07:06.286 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:05 smithi006 sudo[62458]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T12:07:06.287 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:05 smithi006 sudo[62458]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T12:07:06.287 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:05 smithi006 sudo[62458]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T12:07:06.287 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:05 smithi006 sudo[62458]: pam_unix(sudo:session): session closed for user root 2024-03-24T12:07:06.287 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:05 smithi006 ceph-mon[30108]: osdmap e21: 3 total, 3 up, 3 in 2024-03-24T12:07:06.287 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:05 smithi006 ceph-mon[30108]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T12:07:06.287 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:05 smithi006 ceph-mon[41734]: osdmap e21: 3 total, 3 up, 3 in 2024-03-24T12:07:06.287 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:05 smithi006 ceph-mon[41734]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T12:07:06.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:06 smithi090 sudo[36469]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T12:07:06.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:06 smithi090 sudo[36469]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T12:07:06.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:06 smithi090 sudo[36469]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T12:07:06.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:06 smithi090 sudo[36469]: pam_unix(sudo:session): session closed for user root 2024-03-24T12:07:06.956 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:06 smithi006 sudo[62593]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-24T12:07:06.956 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:06 smithi006 sudo[62593]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T12:07:06.956 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:06 smithi006 sudo[62593]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T12:07:06.956 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:06 smithi006 sudo[62593]: pam_unix(sudo:session): session closed for user root 2024-03-24T12:07:07.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[30108]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:07.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[30108]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:07:07.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[30108]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T12:07:07.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T12:07:07.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:07:07.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:07:07.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:07:07.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[30108]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T12:07:07.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[30108]: mgrmap e15: y(active, since 2m), standbys: x 2024-03-24T12:07:07.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:07:07.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T12:07:07.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:07:07.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:07:07.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:07.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[30108]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T12:07:07.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:07.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T12:07:07.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:07:07.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:07:07.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:07 smithi006 sudo[62632]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-24T12:07:07.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:07 smithi006 sudo[62632]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-24T12:07:07.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:07 smithi006 sudo[62632]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-24T12:07:07.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:07 smithi006 sudo[62632]: pam_unix(sudo:session): session closed for user root 2024-03-24T12:07:07.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[41734]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:07.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[41734]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:07:07.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[41734]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T12:07:07.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T12:07:07.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:07:07.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:07:07.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:07:07.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[41734]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T12:07:07.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[41734]: mgrmap e15: y(active, since 2m), standbys: x 2024-03-24T12:07:07.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:07:07.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T12:07:07.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:07:07.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:07:07.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:07.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[41734]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T12:07:07.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:07.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T12:07:07.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:07:07.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:07 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:07:07.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:07 smithi090 ceph-mon[33646]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:07.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:07 smithi090 ceph-mon[33646]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi006:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:07:07.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:07 smithi090 ceph-mon[33646]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T12:07:07.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:07 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T12:07:07.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:07 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:07:07.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:07 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:07:07.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:07 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:07:07.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:07 smithi090 ceph-mon[33646]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T12:07:07.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:07 smithi090 ceph-mon[33646]: mgrmap e15: y(active, since 2m), standbys: x 2024-03-24T12:07:07.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:07 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:07:07.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:07 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T12:07:07.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:07 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:07:07.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:07 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:07:07.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:07 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:07.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:07 smithi090 ceph-mon[33646]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T12:07:07.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:07 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:07.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:07 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T12:07:07.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:07 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:07:07.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:07 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:07:08.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:08 smithi090 ceph-mon[33646]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T12:07:08.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:08 smithi090 ceph-mon[33646]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T12:07:08.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:08 smithi090 ceph-mon[33646]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:08 smithi006 ceph-mon[30108]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T12:07:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:08 smithi006 ceph-mon[30108]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T12:07:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:08 smithi006 ceph-mon[30108]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:08.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:08 smithi006 ceph-mon[41734]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-24T12:07:08.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:08 smithi006 ceph-mon[41734]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-24T12:07:08.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:08 smithi006 ceph-mon[41734]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:09.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:09 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4260939234' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd1c26cc-c121-4570-87ff-f432ecc53e6e"}]: dispatch 2024-03-24T12:07:09.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:09 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4260939234' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd1c26cc-c121-4570-87ff-f432ecc53e6e"}]': finished 2024-03-24T12:07:09.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:09 smithi090 ceph-mon[33646]: osdmap e22: 4 total, 3 up, 4 in 2024-03-24T12:07:09.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:09 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T12:07:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:09 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4260939234' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd1c26cc-c121-4570-87ff-f432ecc53e6e"}]: dispatch 2024-03-24T12:07:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:09 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4260939234' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd1c26cc-c121-4570-87ff-f432ecc53e6e"}]': finished 2024-03-24T12:07:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:09 smithi006 ceph-mon[30108]: osdmap e22: 4 total, 3 up, 4 in 2024-03-24T12:07:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:09 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T12:07:09.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:09 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4260939234' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd1c26cc-c121-4570-87ff-f432ecc53e6e"}]: dispatch 2024-03-24T12:07:09.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:09 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4260939234' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd1c26cc-c121-4570-87ff-f432ecc53e6e"}]': finished 2024-03-24T12:07:09.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:09 smithi006 ceph-mon[41734]: osdmap e22: 4 total, 3 up, 4 in 2024-03-24T12:07:09.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:09 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T12:07:10.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2228947513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:07:10.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:10 smithi090 ceph-mon[33646]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:10.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2228947513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:07:10.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:10 smithi006 ceph-mon[30108]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:10.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2228947513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:07:10.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:10 smithi006 ceph-mon[41734]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:12.150 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:11 smithi006 ceph-mon[30108]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-03-24T12:07:12.151 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:11 smithi006 ceph-mon[41734]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-03-24T12:07:12.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:11 smithi090 ceph-mon[33646]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-03-24T12:07:14.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:13 smithi006 ceph-mon[30108]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-24T12:07:14.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:13 smithi006 ceph-mon[41734]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-24T12:07:14.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:13 smithi090 ceph-mon[33646]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-24T12:07:15.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:15 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T12:07:15.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:15 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:15.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:15 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T12:07:15.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:15 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:15.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:15 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-24T12:07:15.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:15 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:16.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:16 smithi006 ceph-mon[30108]: Deploying daemon osd.3 on smithi006 2024-03-24T12:07:16.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:16 smithi006 ceph-mon[30108]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-24T12:07:16.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:16 smithi006 ceph-mon[41734]: Deploying daemon osd.3 on smithi006 2024-03-24T12:07:16.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:16 smithi006 ceph-mon[41734]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-24T12:07:16.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:16 smithi090 ceph-mon[33646]: Deploying daemon osd.3 on smithi006 2024-03-24T12:07:16.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:16 smithi090 ceph-mon[33646]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-24T12:07:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:17 smithi006 ceph-mon[30108]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-24T12:07:18.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:17 smithi006 ceph-mon[41734]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-24T12:07:18.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:17 smithi090 ceph-mon[33646]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-24T12:07:20.154 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:19 smithi006 ceph-mon[30108]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 71 KiB/s wr, 5 op/s 2024-03-24T12:07:20.154 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:19 smithi006 ceph-mon[41734]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 71 KiB/s wr, 5 op/s 2024-03-24T12:07:20.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:19 smithi090 ceph-mon[33646]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail; 1.6 KiB/s rd, 71 KiB/s wr, 5 op/s 2024-03-24T12:07:22.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:22 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:22.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:22 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:07:22.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:22 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:22.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:22 smithi090 ceph-mon[33646]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-24T12:07:22.439 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:22 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:22.439 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:22 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:07:22.439 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:22 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:22.439 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:22 smithi006 ceph-mon[30108]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-24T12:07:22.439 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:22 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:22.439 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:22 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:07:22.439 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:22 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:22.439 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:22 smithi006 ceph-mon[41734]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-24T12:07:23.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:23 smithi006 ceph-mon[30108]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:23.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:23 smithi006 ceph-mon[41734]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:24.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:23 smithi090 ceph-mon[33646]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:24.643 INFO:teuthology.orchestra.run.smithi006.stdout:Created osd(s) 3 on host 'smithi006' 2024-03-24T12:07:24.644 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:24.643+0000 7f52037fe700 1 -- 172.21.15.6:0/3823433311 <== mgr.14152 v2:172.21.15.6:6800/3847865179 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5214101d40 con 0x7f51fc06d4e0 2024-03-24T12:07:24.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:24.646+0000 7f5218882700 1 -- 172.21.15.6:0/3823433311 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f51fc06d4e0 msgr2=0x7f51fc06f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:07:24.645 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:24.646+0000 7f5218882700 1 --2- 172.21.15.6:0/3823433311 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f51fc06d4e0 0x7f51fc06f9a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f52040059d0 tx=0x7f5204005960 comp rx=0 tx=0).stop 2024-03-24T12:07:24.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:24.646+0000 7f5218882700 1 -- 172.21.15.6:0/3823433311 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5214102f40 msgr2=0x7f5214199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:07:24.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:24.646+0000 7f5218882700 1 --2- 172.21.15.6:0/3823433311 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5214102f40 0x7f5214199920 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f520800e9d0 tx=0x7f520800ece0 comp rx=0 tx=0).stop 2024-03-24T12:07:24.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:24.646+0000 7f5218882700 1 -- 172.21.15.6:0/3823433311 shutdown_connections 2024-03-24T12:07:24.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:24.646+0000 7f5218882700 1 --2- 172.21.15.6:0/3823433311 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f51fc06d4e0 0x7f51fc06f9a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:07:24.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:24.646+0000 7f5218882700 1 --2- 172.21.15.6:0/3823433311 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52141038e0 0x7f52141a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:07:24.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:24.646+0000 7f5218882700 1 --2- 172.21.15.6:0/3823433311 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52140fff60 0x7f52141993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:07:24.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:24.646+0000 7f5218882700 1 --2- 172.21.15.6:0/3823433311 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5214102f40 0x7f5214199920 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:07:24.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:24.646+0000 7f5218882700 1 -- 172.21.15.6:0/3823433311 >> 172.21.15.6:0/3823433311 conn(0x7f52140fb8e0 msgr2=0x7f52140fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:07:24.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:24.646+0000 7f5218882700 1 -- 172.21.15.6:0/3823433311 shutdown_connections 2024-03-24T12:07:24.646 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:07:24.646+0000 7f5218882700 1 -- 172.21.15.6:0/3823433311 wait complete. 2024-03-24T12:07:25.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:24 smithi006 ceph-mon[30108]: from='osd.3 [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T12:07:25.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:24 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:25.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:24 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:25.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:24 smithi006 ceph-mon[41734]: from='osd.3 [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T12:07:25.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:24 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:25.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:24 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:25.327 DEBUG:teuthology.orchestra.run.smithi006:osd.3> sudo journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.3.service 2024-03-24T12:07:25.329 INFO:tasks.cephadm:Deploying osd.4 on smithi090 with /dev/vg_nvme/lv_4... 2024-03-24T12:07:25.330 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-24T12:07:25.359 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:24 smithi090 ceph-mon[33646]: from='osd.3 [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-24T12:07:25.359 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:24 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:25.359 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:24 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:25.575 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T12:07:25.954 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:25 smithi090 ceph-mon[33646]: from='osd.3 [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-24T12:07:25.954 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:25 smithi090 ceph-mon[33646]: osdmap e23: 4 total, 3 up, 4 in 2024-03-24T12:07:25.954 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:25 smithi090 ceph-mon[33646]: from='osd.3 [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-24T12:07:25.954 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:25 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T12:07:25.954 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:25 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:25.954 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:25 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:25.954 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:25 smithi090 ceph-mon[33646]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:26.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:25 smithi006 ceph-mon[30108]: from='osd.3 [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-24T12:07:26.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:25 smithi006 ceph-mon[30108]: osdmap e23: 4 total, 3 up, 4 in 2024-03-24T12:07:26.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:25 smithi006 ceph-mon[30108]: from='osd.3 [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-24T12:07:26.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:25 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T12:07:26.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:25 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:26.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:25 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:26.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:25 smithi006 ceph-mon[30108]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:26.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:25 smithi006 ceph-mon[41734]: from='osd.3 [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-24T12:07:26.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:25 smithi006 ceph-mon[41734]: osdmap e23: 4 total, 3 up, 4 in 2024-03-24T12:07:26.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:25 smithi006 ceph-mon[41734]: from='osd.3 [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]: dispatch 2024-03-24T12:07:26.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:25 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T12:07:26.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:25 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:26.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:25 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:26.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:25 smithi006 ceph-mon[41734]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:26.711 INFO:journalctl@ceph.osd.3.smithi006.stdout:Mar 24 12:07:26 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-3[64519]: 2024-03-24T12:07:26.386+0000 7f30b1ab9700 -1 osd.3 0 waiting for initial osdmap 2024-03-24T12:07:26.711 INFO:journalctl@ceph.osd.3.smithi006.stdout:Mar 24 12:07:26 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-3[64519]: 2024-03-24T12:07:26.391+0000 7f30a98a4700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T12:07:27.162 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:26 smithi006 ceph-mon[30108]: from='osd.3 [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-03-24T12:07:27.162 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:26 smithi006 ceph-mon[30108]: osdmap e24: 4 total, 3 up, 4 in 2024-03-24T12:07:27.162 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:26 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T12:07:27.162 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:26 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T12:07:27.162 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:26 smithi006 ceph-mon[30108]: from='osd.3 [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884]' entity='osd.3' 2024-03-24T12:07:27.162 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:26 smithi006 ceph-mon[41734]: from='osd.3 [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-03-24T12:07:27.162 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:26 smithi006 ceph-mon[41734]: osdmap e24: 4 total, 3 up, 4 in 2024-03-24T12:07:27.163 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:26 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T12:07:27.163 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:26 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T12:07:27.163 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:26 smithi006 ceph-mon[41734]: from='osd.3 [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884]' entity='osd.3' 2024-03-24T12:07:27.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:26 smithi090 ceph-mon[33646]: from='osd.3 [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi006", "root=default"]}]': finished 2024-03-24T12:07:27.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:26 smithi090 ceph-mon[33646]: osdmap e24: 4 total, 3 up, 4 in 2024-03-24T12:07:27.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:26 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T12:07:27.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:26 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T12:07:27.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:26 smithi090 ceph-mon[33646]: from='osd.3 [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884]' entity='osd.3' 2024-03-24T12:07:27.534 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-24T12:07:27.534 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-24T12:07:27.534 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-03-24T12:07:27.534 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-03-24T12:07:27.534 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0140631 s, 746 MB/s 2024-03-24T12:07:27.534 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-03-24T12:07:28.119 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:27 smithi090 ceph-mon[33646]: purged_snaps scrub starts 2024-03-24T12:07:28.119 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:27 smithi090 ceph-mon[33646]: purged_snaps scrub ok 2024-03-24T12:07:28.119 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:27 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T12:07:28.119 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:27 smithi090 ceph-mon[33646]: osd.3 [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884] boot 2024-03-24T12:07:28.119 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:27 smithi090 ceph-mon[33646]: osdmap e25: 4 total, 4 up, 4 in 2024-03-24T12:07:28.119 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:27 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T12:07:28.119 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:27 smithi090 ceph-mon[33646]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:28.145 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi090:vg_nvme/lv_4 2024-03-24T12:07:28.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:27 smithi006 ceph-mon[30108]: purged_snaps scrub starts 2024-03-24T12:07:28.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:27 smithi006 ceph-mon[30108]: purged_snaps scrub ok 2024-03-24T12:07:28.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:27 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T12:07:28.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:27 smithi006 ceph-mon[30108]: osd.3 [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884] boot 2024-03-24T12:07:28.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:27 smithi006 ceph-mon[30108]: osdmap e25: 4 total, 4 up, 4 in 2024-03-24T12:07:28.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:27 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T12:07:28.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:27 smithi006 ceph-mon[30108]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:28.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:27 smithi006 ceph-mon[41734]: purged_snaps scrub starts 2024-03-24T12:07:28.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:27 smithi006 ceph-mon[41734]: purged_snaps scrub ok 2024-03-24T12:07:28.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:27 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T12:07:28.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:27 smithi006 ceph-mon[41734]: osd.3 [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884] boot 2024-03-24T12:07:28.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:27 smithi006 ceph-mon[41734]: osdmap e25: 4 total, 4 up, 4 in 2024-03-24T12:07:28.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:27 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T12:07:28.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:27 smithi006 ceph-mon[41734]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-03-24T12:07:28.426 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T12:07:30.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.104+0000 7f6642279700 1 -- 172.21.15.90:0/1974630326 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f663c105060 msgr2=0x7f663c107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:07:30.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.104+0000 7f6642279700 1 --2- 172.21.15.90:0/1974630326 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f663c105060 0x7f663c107450 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6630009990 tx=0x7f6630009ca0 comp rx=0 tx=0).stop 2024-03-24T12:07:30.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.104+0000 7f6642279700 1 -- 172.21.15.90:0/1974630326 shutdown_connections 2024-03-24T12:07:30.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.104+0000 7f6642279700 1 --2- 172.21.15.90:0/1974630326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f663c107990 0x7f663c109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:07:30.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.104+0000 7f6642279700 1 --2- 172.21.15.90:0/1974630326 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f663c102730 0x7f663c104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:07:30.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.104+0000 7f6642279700 1 --2- 172.21.15.90:0/1974630326 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f663c105060 0x7f663c107450 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:07:30.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.104+0000 7f6642279700 1 -- 172.21.15.90:0/1974630326 >> 172.21.15.90:0/1974630326 conn(0x7f663c0fc140 msgr2=0x7f663c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:07:30.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.104+0000 7f6642279700 1 -- 172.21.15.90:0/1974630326 shutdown_connections 2024-03-24T12:07:30.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.104+0000 7f6642279700 1 -- 172.21.15.90:0/1974630326 wait complete. 2024-03-24T12:07:30.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.104+0000 7f6642279700 1 Processor -- start 2024-03-24T12:07:30.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.105+0000 7f6642279700 1 -- start start 2024-03-24T12:07:30.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.105+0000 7f6642279700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f663c102730 0x7f663c197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:07:30.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.105+0000 7f6642279700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f663c105060 0x7f663c197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:07:30.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.105+0000 7f6642279700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f663c107990 0x7f663c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:07:30.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.105+0000 7f6642279700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c19ee60 con 0x7f663c107990 2024-03-24T12:07:30.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.105+0000 7f6642279700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c19efa0 con 0x7f663c102730 2024-03-24T12:07:30.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.105+0000 7f6642279700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663c19f130 con 0x7f663c105060 2024-03-24T12:07:30.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.105+0000 7f663bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f663c102730 0x7f663c197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:07:30.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.105+0000 7f663bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f663c102730 0x7f663c197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40070/0 (socket says 172.21.15.90:40070) 2024-03-24T12:07:30.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.105+0000 7f663bfff700 1 -- 172.21.15.90:0/2932686848 learned_addr learned my addr 172.21.15.90:0/2932686848 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-24T12:07:30.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.105+0000 7f663bfff700 1 -- 172.21.15.90:0/2932686848 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f663c105060 msgr2=0x7f663c197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:07:30.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.105+0000 7f663bfff700 1 --2- 172.21.15.90:0/2932686848 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f663c105060 0x7f663c197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:07:30.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.105+0000 7f663bfff700 1 -- 172.21.15.90:0/2932686848 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f663c107990 msgr2=0x7f663c19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:07:30.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.105+0000 7f663bfff700 1 --2- 172.21.15.90:0/2932686848 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f663c107990 0x7f663c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:07:30.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.105+0000 7f663bfff700 1 -- 172.21.15.90:0/2932686848 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66300095f0 con 0x7f663c102730 2024-03-24T12:07:30.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.105+0000 7f663bfff700 1 --2- 172.21.15.90:0/2932686848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f663c102730 0x7f663c197320 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f662c00b530 tx=0x7f662c00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:07:30.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.105+0000 7f66397fa700 1 -- 172.21.15.90:0/2932686848 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c011830 con 0x7f663c102730 2024-03-24T12:07:30.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.106+0000 7f6642279700 1 -- 172.21.15.90:0/2932686848 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663c19f410 con 0x7f663c102730 2024-03-24T12:07:30.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.106+0000 7f66397fa700 1 -- 172.21.15.90:0/2932686848 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f662c011e10 con 0x7f663c102730 2024-03-24T12:07:30.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.106+0000 7f66397fa700 1 -- 172.21.15.90:0/2932686848 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c010d80 con 0x7f663c102730 2024-03-24T12:07:30.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.106+0000 7f6642279700 1 -- 172.21.15.90:0/2932686848 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663c19f960 con 0x7f663c102730 2024-03-24T12:07:30.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.106+0000 7f6642279700 1 -- 172.21.15.90:0/2932686848 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663c0fdcc0 con 0x7f663c102730 2024-03-24T12:07:30.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.107+0000 7f66397fa700 1 -- 172.21.15.90:0/2932686848 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 15) v1 ==== 91079+0+0 (secure 0 0 0) 0x7f662c011990 con 0x7f663c102730 2024-03-24T12:07:30.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.107+0000 7f66397fa700 1 --2- 172.21.15.90:0/2932686848 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f662406d570 0x7f662406fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:07:30.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.108+0000 7f66397fa700 1 -- 172.21.15.90:0/2932686848 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3449+0+0 (secure 0 0 0) 0x7f662c08bd60 con 0x7f663c102730 2024-03-24T12:07:30.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.108+0000 7f663b7fe700 1 --2- 172.21.15.90:0/2932686848 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f662406d570 0x7f662406fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:07:30.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.108+0000 7f663b7fe700 1 --2- 172.21.15.90:0/2932686848 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f662406d570 0x7f662406fa30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f66300095c0 tx=0x7f6630009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:07:30.109 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:29 smithi090 ceph-mon[33646]: osdmap e26: 4 total, 4 up, 4 in 2024-03-24T12:07:30.109 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:29 smithi090 ceph-mon[33646]: Detected new or changed devices on smithi006 2024-03-24T12:07:30.109 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:29 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:30.109 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:29 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:30.110 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:29 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:30.110 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:29 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:30.110 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:29 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:30.110 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:29 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:30.110 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:29 smithi090 ceph-mon[33646]: Adjusting osd_memory_target on smithi006 to 4017M 2024-03-24T12:07:30.110 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:29 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:30.110 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:29 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:30.110 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:29 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:07:30.110 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:29 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:30.110 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:29 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:07:30.110 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:29 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:30.110 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:29 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:07:30.110 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:29 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:30.110 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:29 smithi090 ceph-mon[33646]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:30.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.115+0000 7f66397fa700 1 -- 172.21.15.90:0/2932686848 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f662c056390 con 0x7f663c102730 2024-03-24T12:07:30.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[41734]: osdmap e26: 4 total, 4 up, 4 in 2024-03-24T12:07:30.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[41734]: Detected new or changed devices on smithi006 2024-03-24T12:07:30.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:30.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:30.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:30.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:30.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:30.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:30.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[41734]: Adjusting osd_memory_target on smithi006 to 4017M 2024-03-24T12:07:30.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:30.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:30.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:07:30.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:30.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:07:30.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:30.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:07:30.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:30.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[41734]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:30.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[30108]: osdmap e26: 4 total, 4 up, 4 in 2024-03-24T12:07:30.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[30108]: Detected new or changed devices on smithi006 2024-03-24T12:07:30.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:30.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:30.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:30.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:30.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:30.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:30.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[30108]: Adjusting osd_memory_target on smithi006 to 4017M 2024-03-24T12:07:30.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:30.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:30.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:07:30.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:30.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:07:30.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:30.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:07:30.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:30.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:29 smithi006 ceph-mon[30108]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:30.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:30.314+0000 7f6642279700 1 -- 172.21.15.90:0/2932686848 --> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f663c0620b0 con 0x7f662406d570 2024-03-24T12:07:31.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:30 smithi090 ceph-mon[33646]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:07:31.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:30 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:07:31.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:30 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:07:31.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:30 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:31.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:30 smithi006 ceph-mon[30108]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:07:31.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:30 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:07:31.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:30 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:07:31.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:30 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:31.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:30 smithi006 ceph-mon[41734]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:07:31.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:30 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:07:31.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:30 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:07:31.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:30 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:32.269 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:31 smithi090 ceph-mon[33646]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:31 smithi006 ceph-mon[30108]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:32.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:31 smithi006 ceph-mon[41734]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:33.355 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.90:0/1439243266' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65a98356-0941-41e8-a10b-9d0f76212fd1"}]: dispatch 2024-03-24T12:07:33.355 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:33 smithi090 ceph-mon[33646]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65a98356-0941-41e8-a10b-9d0f76212fd1"}]: dispatch 2024-03-24T12:07:33.355 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:33 smithi090 ceph-mon[33646]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65a98356-0941-41e8-a10b-9d0f76212fd1"}]': finished 2024-03-24T12:07:33.355 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:33 smithi090 ceph-mon[33646]: osdmap e27: 5 total, 4 up, 5 in 2024-03-24T12:07:33.355 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:33 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T12:07:33.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.90:0/1439243266' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65a98356-0941-41e8-a10b-9d0f76212fd1"}]: dispatch 2024-03-24T12:07:33.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:33 smithi006 ceph-mon[41734]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65a98356-0941-41e8-a10b-9d0f76212fd1"}]: dispatch 2024-03-24T12:07:33.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:33 smithi006 ceph-mon[41734]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65a98356-0941-41e8-a10b-9d0f76212fd1"}]': finished 2024-03-24T12:07:33.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:33 smithi006 ceph-mon[41734]: osdmap e27: 5 total, 4 up, 5 in 2024-03-24T12:07:33.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:33 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T12:07:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.90:0/1439243266' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65a98356-0941-41e8-a10b-9d0f76212fd1"}]: dispatch 2024-03-24T12:07:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:33 smithi006 ceph-mon[30108]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65a98356-0941-41e8-a10b-9d0f76212fd1"}]: dispatch 2024-03-24T12:07:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:33 smithi006 ceph-mon[30108]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65a98356-0941-41e8-a10b-9d0f76212fd1"}]': finished 2024-03-24T12:07:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:33 smithi006 ceph-mon[30108]: osdmap e27: 5 total, 4 up, 5 in 2024-03-24T12:07:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:33 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T12:07:34.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.90:0/3568959441' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:07:34.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:34 smithi090 ceph-mon[33646]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:34.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.90:0/3568959441' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:07:34.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:34 smithi006 ceph-mon[30108]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:34.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.90:0/3568959441' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:07:34.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:34 smithi006 ceph-mon[41734]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:36.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:35 smithi090 ceph-mon[33646]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:36.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:35 smithi006 ceph-mon[30108]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:36.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:35 smithi006 ceph-mon[41734]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:38.035 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:37 smithi090 ceph-mon[33646]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:38.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:37 smithi006 ceph-mon[41734]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:38.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:37 smithi006 ceph-mon[30108]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:39.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:39 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T12:07:39.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:39 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:39.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:39 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T12:07:39.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:39 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:39.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:39 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-24T12:07:39.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:39 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:40.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:40 smithi090 ceph-mon[33646]: Deploying daemon osd.4 on smithi090 2024-03-24T12:07:40.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:40 smithi090 ceph-mon[33646]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:40.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:40 smithi006 ceph-mon[30108]: Deploying daemon osd.4 on smithi090 2024-03-24T12:07:40.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:40 smithi006 ceph-mon[30108]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:40.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:40 smithi006 ceph-mon[41734]: Deploying daemon osd.4 on smithi090 2024-03-24T12:07:40.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:40 smithi006 ceph-mon[41734]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:42.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:41 smithi006 ceph-mon[30108]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:42.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:41 smithi006 ceph-mon[41734]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:42.213 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:41 smithi090 ceph-mon[33646]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:44.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:43 smithi006 ceph-mon[30108]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:44.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:43 smithi006 ceph-mon[41734]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:43 smithi090 ceph-mon[33646]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:46.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:45 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:46.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:45 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:46.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:45 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:07:46.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:45 smithi090 ceph-mon[33646]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:46.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:46.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:46.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:07:46.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:45 smithi006 ceph-mon[30108]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:46.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:45 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:46.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:45 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:46.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:45 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:07:46.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:45 smithi006 ceph-mon[41734]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:47 smithi006 ceph-mon[30108]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:48.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:47 smithi006 ceph-mon[41734]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:48.222 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:47 smithi090 ceph-mon[33646]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:48.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:48.275+0000 7f66397fa700 1 -- 172.21.15.90:0/2932686848 <== mgr.14152 v2:172.21.15.6:6800/3847865179 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f663c0620b0 con 0x7f662406d570 2024-03-24T12:07:48.275 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 4 on host 'smithi090' 2024-03-24T12:07:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:48.277+0000 7f6642279700 1 -- 172.21.15.90:0/2932686848 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f662406d570 msgr2=0x7f662406fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:07:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:48.277+0000 7f6642279700 1 --2- 172.21.15.90:0/2932686848 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f662406d570 0x7f662406fa30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f66300095c0 tx=0x7f6630009530 comp rx=0 tx=0).stop 2024-03-24T12:07:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:48.277+0000 7f6642279700 1 -- 172.21.15.90:0/2932686848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f663c102730 msgr2=0x7f663c197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:07:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:48.277+0000 7f6642279700 1 --2- 172.21.15.90:0/2932686848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f663c102730 0x7f663c197320 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f662c00b530 tx=0x7f662c00b8f0 comp rx=0 tx=0).stop 2024-03-24T12:07:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:48.278+0000 7f6642279700 1 -- 172.21.15.90:0/2932686848 shutdown_connections 2024-03-24T12:07:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:48.278+0000 7f6642279700 1 --2- 172.21.15.90:0/2932686848 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f662406d570 0x7f662406fa30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:07:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:48.278+0000 7f6642279700 1 --2- 172.21.15.90:0/2932686848 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f663c107990 0x7f663c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:07:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:48.278+0000 7f6642279700 1 --2- 172.21.15.90:0/2932686848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f663c102730 0x7f663c197320 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:07:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:48.278+0000 7f6642279700 1 --2- 172.21.15.90:0/2932686848 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f663c105060 0x7f663c197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:07:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:48.278+0000 7f6642279700 1 -- 172.21.15.90:0/2932686848 >> 172.21.15.90:0/2932686848 conn(0x7f663c0fc140 msgr2=0x7f663c1033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:07:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:48.278+0000 7f6642279700 1 -- 172.21.15.90:0/2932686848 shutdown_connections 2024-03-24T12:07:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:48.278+0000 7f6642279700 1 -- 172.21.15.90:0/2932686848 wait complete. 2024-03-24T12:07:48.906 DEBUG:teuthology.orchestra.run.smithi090:osd.4> sudo journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.4.service 2024-03-24T12:07:48.909 INFO:tasks.cephadm:Deploying osd.5 on smithi090 with /dev/vg_nvme/lv_3... 2024-03-24T12:07:48.909 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-24T12:07:49.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:48 smithi090 ceph-mon[33646]: from='osd.4 [v2:172.21.15.90:6800/3147977284,v1:172.21.15.90:6801/3147977284]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T12:07:49.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:48 smithi090 ceph-mon[33646]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T12:07:49.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:48 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:49.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:48 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:49.194 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T12:07:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:48 smithi006 ceph-mon[30108]: from='osd.4 [v2:172.21.15.90:6800/3147977284,v1:172.21.15.90:6801/3147977284]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T12:07:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:48 smithi006 ceph-mon[30108]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T12:07:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:48 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:48 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:49.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:48 smithi006 ceph-mon[41734]: from='osd.4 [v2:172.21.15.90:6800/3147977284,v1:172.21.15.90:6801/3147977284]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T12:07:49.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:48 smithi006 ceph-mon[41734]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-24T12:07:49.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:48 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:49.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:48 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:50.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:49 smithi006 ceph-mon[30108]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-24T12:07:50.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:49 smithi006 ceph-mon[30108]: osdmap e28: 5 total, 4 up, 5 in 2024-03-24T12:07:50.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:49 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T12:07:50.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:49 smithi006 ceph-mon[30108]: from='osd.4 [v2:172.21.15.90:6800/3147977284,v1:172.21.15.90:6801/3147977284]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:07:50.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:49 smithi006 ceph-mon[30108]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:07:50.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:49 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:50.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:49 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:50.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:49 smithi006 ceph-mon[30108]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:50.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:49 smithi006 ceph-mon[41734]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-24T12:07:50.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:49 smithi006 ceph-mon[41734]: osdmap e28: 5 total, 4 up, 5 in 2024-03-24T12:07:50.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:49 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T12:07:50.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:49 smithi006 ceph-mon[41734]: from='osd.4 [v2:172.21.15.90:6800/3147977284,v1:172.21.15.90:6801/3147977284]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:07:50.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:49 smithi006 ceph-mon[41734]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:07:50.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:49 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:50.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:49 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:50.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:49 smithi006 ceph-mon[41734]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:50.236 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:49 smithi090 ceph-mon[33646]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-24T12:07:50.236 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:49 smithi090 ceph-mon[33646]: osdmap e28: 5 total, 4 up, 5 in 2024-03-24T12:07:50.236 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:49 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T12:07:50.236 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:49 smithi090 ceph-mon[33646]: from='osd.4 [v2:172.21.15.90:6800/3147977284,v1:172.21.15.90:6801/3147977284]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:07:50.236 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:49 smithi090 ceph-mon[33646]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:07:50.236 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:49 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:50.236 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:49 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:50.236 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:49 smithi090 ceph-mon[33646]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-24T12:07:50.237 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:07:49 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4[39167]: 2024-03-24T12:07:49.957+0000 7fb7e1de6700 -1 osd.4 0 waiting for initial osdmap 2024-03-24T12:07:50.237 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:07:49 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4[39167]: 2024-03-24T12:07:49.960+0000 7fb7ddbd9700 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T12:07:51.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:50 smithi090 ceph-mon[33646]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-03-24T12:07:51.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:50 smithi090 ceph-mon[33646]: osdmap e29: 5 total, 4 up, 5 in 2024-03-24T12:07:51.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:50 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T12:07:51.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:50 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T12:07:51.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:50 smithi090 ceph-mon[33646]: osd.4 [v2:172.21.15.90:6800/3147977284,v1:172.21.15.90:6801/3147977284] boot 2024-03-24T12:07:51.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:50 smithi090 ceph-mon[33646]: osdmap e30: 5 total, 5 up, 5 in 2024-03-24T12:07:51.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:50 smithi006 ceph-mon[30108]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-03-24T12:07:51.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:50 smithi006 ceph-mon[30108]: osdmap e29: 5 total, 4 up, 5 in 2024-03-24T12:07:51.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:50 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T12:07:51.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:50 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T12:07:51.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:50 smithi006 ceph-mon[30108]: osd.4 [v2:172.21.15.90:6800/3147977284,v1:172.21.15.90:6801/3147977284] boot 2024-03-24T12:07:51.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:50 smithi006 ceph-mon[30108]: osdmap e30: 5 total, 5 up, 5 in 2024-03-24T12:07:51.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:50 smithi006 ceph-mon[41734]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-03-24T12:07:51.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:50 smithi006 ceph-mon[41734]: osdmap e29: 5 total, 4 up, 5 in 2024-03-24T12:07:51.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:50 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T12:07:51.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:50 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T12:07:51.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:50 smithi006 ceph-mon[41734]: osd.4 [v2:172.21.15.90:6800/3147977284,v1:172.21.15.90:6801/3147977284] boot 2024-03-24T12:07:51.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:50 smithi006 ceph-mon[41734]: osdmap e30: 5 total, 5 up, 5 in 2024-03-24T12:07:51.315 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-24T12:07:51.315 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-24T12:07:51.315 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-03-24T12:07:51.315 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-03-24T12:07:51.315 INFO:teuthology.orchestra.run.smithi090.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0141046 s, 743 MB/s 2024-03-24T12:07:51.315 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-03-24T12:07:51.969 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi090:vg_nvme/lv_3 2024-03-24T12:07:52.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:51 smithi006 ceph-mon[30108]: purged_snaps scrub starts 2024-03-24T12:07:52.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:51 smithi006 ceph-mon[30108]: purged_snaps scrub ok 2024-03-24T12:07:52.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:51 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T12:07:52.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:51 smithi006 ceph-mon[30108]: osdmap e31: 5 total, 5 up, 5 in 2024-03-24T12:07:52.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:51 smithi006 ceph-mon[30108]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:07:52.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:51 smithi006 ceph-mon[41734]: purged_snaps scrub starts 2024-03-24T12:07:52.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:51 smithi006 ceph-mon[41734]: purged_snaps scrub ok 2024-03-24T12:07:52.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:51 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T12:07:52.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:51 smithi006 ceph-mon[41734]: osdmap e31: 5 total, 5 up, 5 in 2024-03-24T12:07:52.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:51 smithi006 ceph-mon[41734]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:07:52.335 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:51 smithi090 ceph-mon[33646]: purged_snaps scrub starts 2024-03-24T12:07:52.335 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:51 smithi090 ceph-mon[33646]: purged_snaps scrub ok 2024-03-24T12:07:52.336 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:51 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T12:07:52.336 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:51 smithi090 ceph-mon[33646]: osdmap e31: 5 total, 5 up, 5 in 2024-03-24T12:07:52.336 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:51 smithi090 ceph-mon[33646]: pgmap v114: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:07:52.862 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T12:07:53.585 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:53 smithi090 ceph-mon[33646]: osdmap e32: 5 total, 5 up, 5 in 2024-03-24T12:07:53.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:53 smithi006 ceph-mon[30108]: osdmap e32: 5 total, 5 up, 5 in 2024-03-24T12:07:53.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:53 smithi006 ceph-mon[41734]: osdmap e32: 5 total, 5 up, 5 in 2024-03-24T12:07:54.822 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:54 smithi090 ceph-mon[33646]: osdmap e33: 5 total, 5 up, 5 in 2024-03-24T12:07:54.822 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:54 smithi090 ceph-mon[33646]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:07:54.822 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:54 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:54.822 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:54 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:54.823 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:54 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:54.823 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:54 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:54.823 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:54 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:54.823 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:54 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:07:54.823 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:54 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:54.823 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:54 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:07:54.823 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:54 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:54.823 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:54 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:07:54.823 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:54 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:54.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[30108]: osdmap e33: 5 total, 5 up, 5 in 2024-03-24T12:07:54.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[30108]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:07:54.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:54.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:54.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:54.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:54.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:54.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:07:54.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:54.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:07:54.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:54.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:07:54.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:54.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[41734]: osdmap e33: 5 total, 5 up, 5 in 2024-03-24T12:07:54.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[41734]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:07:54.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:54.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:54.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:07:54.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:54.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:54.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:07:54.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:54.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:07:54.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:54.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:07:54.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:54 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:07:55.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.149+0000 7fb46463c700 1 -- 172.21.15.90:0/2605113188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb45c100bd0 msgr2=0x7fb45c100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:07:55.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.149+0000 7fb46463c700 1 --2- 172.21.15.90:0/2605113188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb45c100bd0 0x7fb45c100ff0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb4580099c0 tx=0x7fb458009cd0 comp rx=0 tx=0).stop 2024-03-24T12:07:55.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.149+0000 7fb46463c700 1 -- 172.21.15.90:0/2605113188 shutdown_connections 2024-03-24T12:07:55.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.149+0000 7fb46463c700 1 --2- 172.21.15.90:0/2605113188 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb45c102770 0x7fb45c109020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:07:55.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.149+0000 7fb46463c700 1 --2- 172.21.15.90:0/2605113188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb45c100bd0 0x7fb45c100ff0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:07:55.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.149+0000 7fb46463c700 1 --2- 172.21.15.90:0/2605113188 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb45c101dd0 0x7fb45c102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:07:55.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.149+0000 7fb46463c700 1 -- 172.21.15.90:0/2605113188 >> 172.21.15.90:0/2605113188 conn(0x7fb45c0fc190 msgr2=0x7fb45c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:07:55.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.149+0000 7fb46463c700 1 -- 172.21.15.90:0/2605113188 shutdown_connections 2024-03-24T12:07:55.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.149+0000 7fb46463c700 1 -- 172.21.15.90:0/2605113188 wait complete. 2024-03-24T12:07:55.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.150+0000 7fb46463c700 1 Processor -- start 2024-03-24T12:07:55.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.150+0000 7fb46463c700 1 -- start start 2024-03-24T12:07:55.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.150+0000 7fb46463c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb45c100bd0 0x7fb45c10b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:07:55.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.150+0000 7fb46463c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb45c101dd0 0x7fb45c109b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:07:55.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.150+0000 7fb46463c700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb45c102770 0x7fb45c10a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:07:55.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.150+0000 7fb46463c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c10a7f0 con 0x7fb45c100bd0 2024-03-24T12:07:55.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.150+0000 7fb46463c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c10a960 con 0x7fb45c101dd0 2024-03-24T12:07:55.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.150+0000 7fb46463c700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c1a1330 con 0x7fb45c102770 2024-03-24T12:07:55.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.150+0000 7fb461bd7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb45c101dd0 0x7fb45c109b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:07:55.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.150+0000 7fb461bd7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb45c101dd0 0x7fb45c109b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55336/0 (socket says 172.21.15.90:55336) 2024-03-24T12:07:55.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.150+0000 7fb461bd7700 1 -- 172.21.15.90:0/3235085773 learned_addr learned my addr 172.21.15.90:0/3235085773 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-24T12:07:55.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.150+0000 7fb462bd9700 1 --2- 172.21.15.90:0/3235085773 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb45c102770 0x7fb45c10a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:07:55.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.150+0000 7fb461bd7700 1 -- 172.21.15.90:0/3235085773 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb45c102770 msgr2=0x7fb45c10a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:07:55.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.150+0000 7fb461bd7700 1 --2- 172.21.15.90:0/3235085773 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb45c102770 0x7fb45c10a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:07:55.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.150+0000 7fb4623d8700 1 --2- 172.21.15.90:0/3235085773 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb45c100bd0 0x7fb45c10b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:07:55.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.150+0000 7fb461bd7700 1 -- 172.21.15.90:0/3235085773 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb45c100bd0 msgr2=0x7fb45c10b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:07:55.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.150+0000 7fb461bd7700 1 --2- 172.21.15.90:0/3235085773 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb45c100bd0 0x7fb45c10b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:07:55.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.151+0000 7fb461bd7700 1 -- 172.21.15.90:0/3235085773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4580095f0 con 0x7fb45c101dd0 2024-03-24T12:07:55.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.151+0000 7fb461bd7700 1 --2- 172.21.15.90:0/3235085773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb45c101dd0 0x7fb45c109b30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb45000b530 tx=0x7fb45000b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:07:55.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.151+0000 7fb44f7fe700 1 -- 172.21.15.90:0/3235085773 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb450011830 con 0x7fb45c101dd0 2024-03-24T12:07:55.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.151+0000 7fb44f7fe700 1 -- 172.21.15.90:0/3235085773 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fb450011e10 con 0x7fb45c101dd0 2024-03-24T12:07:55.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.151+0000 7fb46463c700 1 -- 172.21.15.90:0/3235085773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45c1a1610 con 0x7fb45c101dd0 2024-03-24T12:07:55.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.151+0000 7fb44f7fe700 1 -- 172.21.15.90:0/3235085773 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb450010d50 con 0x7fb45c101dd0 2024-03-24T12:07:55.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.151+0000 7fb46463c700 1 -- 172.21.15.90:0/3235085773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb45c1a1be0 con 0x7fb45c101dd0 2024-03-24T12:07:55.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.152+0000 7fb46463c700 1 -- 172.21.15.90:0/3235085773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb45c1077b0 con 0x7fb45c101dd0 2024-03-24T12:07:55.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.152+0000 7fb44f7fe700 1 -- 172.21.15.90:0/3235085773 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 15) v1 ==== 91079+0+0 (secure 0 0 0) 0x7fb450011990 con 0x7fb45c101dd0 2024-03-24T12:07:55.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.153+0000 7fb44f7fe700 1 --2- 172.21.15.90:0/3235085773 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fb44806d570 0x7fb44806fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:07:55.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.153+0000 7fb44f7fe700 1 -- 172.21.15.90:0/3235085773 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 3991+0+0 (secure 0 0 0) 0x7fb45008bfa0 con 0x7fb45c101dd0 2024-03-24T12:07:55.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.153+0000 7fb4623d8700 1 --2- 172.21.15.90:0/3235085773 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fb44806d570 0x7fb44806fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:07:55.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.154+0000 7fb4623d8700 1 --2- 172.21.15.90:0/3235085773 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fb44806d570 0x7fb44806fa30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb458006010 tx=0x7fb458000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:07:55.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.158+0000 7fb44f7fe700 1 -- 172.21.15.90:0/3235085773 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb4500bfd20 con 0x7fb45c101dd0 2024-03-24T12:07:55.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:07:55.357+0000 7fb46463c700 1 -- 172.21.15.90:0/3235085773 --> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fb45c0620b0 con 0x7fb44806d570 2024-03-24T12:07:55.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:55 smithi090 ceph-mon[33646]: Detected new or changed devices on smithi090 2024-03-24T12:07:55.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:55 smithi090 ceph-mon[33646]: Adjusting osd_memory_target on smithi090 to 17088M 2024-03-24T12:07:55.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:55 smithi090 ceph-mon[33646]: osdmap e34: 5 total, 5 up, 5 in 2024-03-24T12:07:55.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:55 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:07:55.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:55 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:07:55.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:55 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:55.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:55 smithi006 ceph-mon[30108]: Detected new or changed devices on smithi090 2024-03-24T12:07:55.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:55 smithi006 ceph-mon[30108]: Adjusting osd_memory_target on smithi090 to 17088M 2024-03-24T12:07:55.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:55 smithi006 ceph-mon[30108]: osdmap e34: 5 total, 5 up, 5 in 2024-03-24T12:07:55.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:55 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:07:55.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:55 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:07:55.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:55 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:55.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:55 smithi006 ceph-mon[41734]: Detected new or changed devices on smithi090 2024-03-24T12:07:55.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:55 smithi006 ceph-mon[41734]: Adjusting osd_memory_target on smithi090 to 17088M 2024-03-24T12:07:55.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:55 smithi006 ceph-mon[41734]: osdmap e34: 5 total, 5 up, 5 in 2024-03-24T12:07:55.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:55 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:07:55.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:55 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:07:55.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:55 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:07:56.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:56 smithi090 ceph-mon[33646]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:07:56.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:56 smithi090 ceph-mon[33646]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:07:56.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:56 smithi006 ceph-mon[30108]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:07:56.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:56 smithi006 ceph-mon[30108]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:07:56.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:56 smithi006 ceph-mon[41734]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:07:56.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:56 smithi006 ceph-mon[41734]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:07:58.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:57 smithi006 ceph-mon[30108]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-24T12:07:58.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:57 smithi006 ceph-mon[41734]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-24T12:07:58.327 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:57 smithi090 ceph-mon[33646]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-24T12:07:59.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:58 smithi006 ceph-mon[30108]: from='client.? 172.21.15.90:0/1376389654' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b143b33c-9447-4b8c-9991-5037f50789ca"}]: dispatch 2024-03-24T12:07:59.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:58 smithi006 ceph-mon[30108]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b143b33c-9447-4b8c-9991-5037f50789ca"}]: dispatch 2024-03-24T12:07:59.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:58 smithi006 ceph-mon[30108]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b143b33c-9447-4b8c-9991-5037f50789ca"}]': finished 2024-03-24T12:07:59.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:58 smithi006 ceph-mon[30108]: osdmap e35: 6 total, 5 up, 6 in 2024-03-24T12:07:59.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:58 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T12:07:59.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:58 smithi006 ceph-mon[30108]: from='client.? 172.21.15.90:0/37934332' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:07:59.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:58 smithi006 ceph-mon[41734]: from='client.? 172.21.15.90:0/1376389654' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b143b33c-9447-4b8c-9991-5037f50789ca"}]: dispatch 2024-03-24T12:07:59.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:58 smithi006 ceph-mon[41734]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b143b33c-9447-4b8c-9991-5037f50789ca"}]: dispatch 2024-03-24T12:07:59.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:58 smithi006 ceph-mon[41734]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b143b33c-9447-4b8c-9991-5037f50789ca"}]': finished 2024-03-24T12:07:59.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:58 smithi006 ceph-mon[41734]: osdmap e35: 6 total, 5 up, 6 in 2024-03-24T12:07:59.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:58 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T12:07:59.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:58 smithi006 ceph-mon[41734]: from='client.? 172.21.15.90:0/37934332' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:07:59.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:58 smithi090 ceph-mon[33646]: from='client.? 172.21.15.90:0/1376389654' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b143b33c-9447-4b8c-9991-5037f50789ca"}]: dispatch 2024-03-24T12:07:59.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:58 smithi090 ceph-mon[33646]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b143b33c-9447-4b8c-9991-5037f50789ca"}]: dispatch 2024-03-24T12:07:59.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:58 smithi090 ceph-mon[33646]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b143b33c-9447-4b8c-9991-5037f50789ca"}]': finished 2024-03-24T12:07:59.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:58 smithi090 ceph-mon[33646]: osdmap e35: 6 total, 5 up, 6 in 2024-03-24T12:07:59.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:58 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T12:07:59.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:58 smithi090 ceph-mon[33646]: from='client.? 172.21.15.90:0/37934332' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:08:00.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:07:59 smithi006 ceph-mon[30108]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-03-24T12:08:00.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:07:59 smithi006 ceph-mon[41734]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-03-24T12:08:00.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:07:59 smithi090 ceph-mon[33646]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-03-24T12:08:02.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:01 smithi006 ceph-mon[30108]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-24T12:08:02.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:01 smithi006 ceph-mon[41734]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-24T12:08:02.261 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:01 smithi090 ceph-mon[33646]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-24T12:08:04.029 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:03 smithi090 ceph-mon[33646]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-03-24T12:08:04.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:03 smithi006 ceph-mon[30108]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-03-24T12:08:04.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:03 smithi006 ceph-mon[41734]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-03-24T12:08:05.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:04 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T12:08:05.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:04 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:05.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:04 smithi006 ceph-mon[30108]: Deploying daemon osd.5 on smithi090 2024-03-24T12:08:05.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:04 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T12:08:05.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:04 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:05.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:04 smithi006 ceph-mon[41734]: Deploying daemon osd.5 on smithi090 2024-03-24T12:08:05.309 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:04 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-24T12:08:05.309 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:04 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:05.309 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:04 smithi090 ceph-mon[33646]: Deploying daemon osd.5 on smithi090 2024-03-24T12:08:06.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:05 smithi006 ceph-mon[30108]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-24T12:08:06.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:05 smithi006 ceph-mon[41734]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-24T12:08:06.313 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:05 smithi090 ceph-mon[33646]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-24T12:08:08.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:07 smithi090 ceph-mon[33646]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:08:08.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:07 smithi006 ceph-mon[30108]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:08:08.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:07 smithi006 ceph-mon[41734]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:08:10.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:09 smithi006 ceph-mon[30108]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:08:10.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:09 smithi006 ceph-mon[41734]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:08:10.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:09 smithi090 ceph-mon[33646]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:08:11.129 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:11 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:11.129 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:11 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:11.129 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:11 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:08:11.129 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:11 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:11.129 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:11 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:11.130 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:11 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:08:11.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:11 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:11.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:11 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:11.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:11 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:08:12.386 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:12 smithi090 ceph-mon[33646]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:08:12.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:12 smithi006 ceph-mon[41734]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:08:12.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:12 smithi006 ceph-mon[30108]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:08:13.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:13 smithi090 ceph-mon[33646]: from='osd.5 [v2:172.21.15.90:6808/3585923902,v1:172.21.15.90:6809/3585923902]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T12:08:13.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:13 smithi090 ceph-mon[33646]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T12:08:13.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:13.524+0000 7fb44f7fe700 1 -- 172.21.15.90:0/3235085773 <== mgr.14152 v2:172.21.15.6:6800/3847865179 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb45c0620b0 con 0x7fb44806d570 2024-03-24T12:08:13.523 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 5 on host 'smithi090' 2024-03-24T12:08:13.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:13.527+0000 7fb46463c700 1 -- 172.21.15.90:0/3235085773 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fb44806d570 msgr2=0x7fb44806fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:08:13.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:13.527+0000 7fb46463c700 1 --2- 172.21.15.90:0/3235085773 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fb44806d570 0x7fb44806fa30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb458006010 tx=0x7fb458000bc0 comp rx=0 tx=0).stop 2024-03-24T12:08:13.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:13.527+0000 7fb46463c700 1 -- 172.21.15.90:0/3235085773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb45c101dd0 msgr2=0x7fb45c109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:08:13.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:13.527+0000 7fb46463c700 1 --2- 172.21.15.90:0/3235085773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb45c101dd0 0x7fb45c109b30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb45000b530 tx=0x7fb45000b8f0 comp rx=0 tx=0).stop 2024-03-24T12:08:13.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:13.527+0000 7fb46463c700 1 -- 172.21.15.90:0/3235085773 shutdown_connections 2024-03-24T12:08:13.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:13.527+0000 7fb46463c700 1 --2- 172.21.15.90:0/3235085773 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fb44806d570 0x7fb44806fa30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:08:13.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:13.527+0000 7fb46463c700 1 --2- 172.21.15.90:0/3235085773 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb45c102770 0x7fb45c10a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:08:13.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:13.527+0000 7fb46463c700 1 --2- 172.21.15.90:0/3235085773 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb45c100bd0 0x7fb45c10b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:08:13.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:13.527+0000 7fb46463c700 1 --2- 172.21.15.90:0/3235085773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb45c101dd0 0x7fb45c109b30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:08:13.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:13.527+0000 7fb46463c700 1 -- 172.21.15.90:0/3235085773 >> 172.21.15.90:0/3235085773 conn(0x7fb45c0fc190 msgr2=0x7fb45c0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:08:13.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:13.527+0000 7fb46463c700 1 -- 172.21.15.90:0/3235085773 shutdown_connections 2024-03-24T12:08:13.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:13.527+0000 7fb46463c700 1 -- 172.21.15.90:0/3235085773 wait complete. 2024-03-24T12:08:13.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:13 smithi006 ceph-mon[30108]: from='osd.5 [v2:172.21.15.90:6808/3585923902,v1:172.21.15.90:6809/3585923902]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T12:08:13.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:13 smithi006 ceph-mon[30108]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T12:08:13.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:13 smithi006 ceph-mon[41734]: from='osd.5 [v2:172.21.15.90:6808/3585923902,v1:172.21.15.90:6809/3585923902]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T12:08:13.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:13 smithi006 ceph-mon[41734]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-24T12:08:14.159 DEBUG:teuthology.orchestra.run.smithi090:osd.5> sudo journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.5.service 2024-03-24T12:08:14.162 INFO:tasks.cephadm:Deploying osd.6 on smithi090 with /dev/vg_nvme/lv_2... 2024-03-24T12:08:14.162 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-24T12:08:14.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:14 smithi090 ceph-mon[33646]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-24T12:08:14.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:14 smithi090 ceph-mon[33646]: from='osd.5 [v2:172.21.15.90:6808/3585923902,v1:172.21.15.90:6809/3585923902]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:08:14.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:14 smithi090 ceph-mon[33646]: osdmap e36: 6 total, 5 up, 6 in 2024-03-24T12:08:14.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:14 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T12:08:14.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:14 smithi090 ceph-mon[33646]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:08:14.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:14 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:14.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:14 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:14.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:14 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:14.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:14 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:14.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:14 smithi090 ceph-mon[33646]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:08:14.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:14 smithi090 ceph-mon[33646]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-03-24T12:08:14.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:14 smithi090 ceph-mon[33646]: osdmap e37: 6 total, 5 up, 6 in 2024-03-24T12:08:14.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[41734]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-24T12:08:14.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[41734]: from='osd.5 [v2:172.21.15.90:6808/3585923902,v1:172.21.15.90:6809/3585923902]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:08:14.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[41734]: osdmap e36: 6 total, 5 up, 6 in 2024-03-24T12:08:14.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T12:08:14.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[41734]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:08:14.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:14.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:14.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:14.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:14.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[41734]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:08:14.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[41734]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-03-24T12:08:14.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[41734]: osdmap e37: 6 total, 5 up, 6 in 2024-03-24T12:08:14.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[30108]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-24T12:08:14.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[30108]: from='osd.5 [v2:172.21.15.90:6808/3585923902,v1:172.21.15.90:6809/3585923902]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:08:14.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[30108]: osdmap e36: 6 total, 5 up, 6 in 2024-03-24T12:08:14.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T12:08:14.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[30108]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:08:14.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:14.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:14.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:14.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:14.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[30108]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:08:14.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[30108]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-03-24T12:08:14.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:14 smithi006 ceph-mon[30108]: osdmap e37: 6 total, 5 up, 6 in 2024-03-24T12:08:14.719 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:08:14 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:08:14.691+0000 7fdb05c14700 -1 osd.5 0 waiting for initial osdmap 2024-03-24T12:08:14.719 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:08:14 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:08:14.697+0000 7fdafd1fe700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T12:08:15.214 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T12:08:15.583 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:15 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T12:08:15.583 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:15 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T12:08:15.583 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:15 smithi090 ceph-mon[33646]: from='osd.5 ' entity='osd.5' 2024-03-24T12:08:15.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:15 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T12:08:15.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:15 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T12:08:15.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:15 smithi006 ceph-mon[30108]: from='osd.5 ' entity='osd.5' 2024-03-24T12:08:15.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:15 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T12:08:15.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:15 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T12:08:15.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:15 smithi006 ceph-mon[41734]: from='osd.5 ' entity='osd.5' 2024-03-24T12:08:16.607 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:16 smithi090 ceph-mon[33646]: purged_snaps scrub starts 2024-03-24T12:08:16.607 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:16 smithi090 ceph-mon[33646]: purged_snaps scrub ok 2024-03-24T12:08:16.607 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:16 smithi090 ceph-mon[33646]: osd.5 [v2:172.21.15.90:6808/3585923902,v1:172.21.15.90:6809/3585923902] boot 2024-03-24T12:08:16.608 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:16 smithi090 ceph-mon[33646]: osdmap e38: 6 total, 6 up, 6 in 2024-03-24T12:08:16.608 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:16 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T12:08:16.608 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:16 smithi090 ceph-mon[33646]: pgmap v133: 1 pgs: 1 remapped; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:08:16.608 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:16 smithi090 ceph-mon[33646]: osdmap e39: 6 total, 6 up, 6 in 2024-03-24T12:08:16.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:16 smithi006 ceph-mon[30108]: purged_snaps scrub starts 2024-03-24T12:08:16.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:16 smithi006 ceph-mon[30108]: purged_snaps scrub ok 2024-03-24T12:08:16.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:16 smithi006 ceph-mon[30108]: osd.5 [v2:172.21.15.90:6808/3585923902,v1:172.21.15.90:6809/3585923902] boot 2024-03-24T12:08:16.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:16 smithi006 ceph-mon[30108]: osdmap e38: 6 total, 6 up, 6 in 2024-03-24T12:08:16.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:16 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T12:08:16.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:16 smithi006 ceph-mon[30108]: pgmap v133: 1 pgs: 1 remapped; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:08:16.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:16 smithi006 ceph-mon[30108]: osdmap e39: 6 total, 6 up, 6 in 2024-03-24T12:08:16.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:16 smithi006 ceph-mon[41734]: purged_snaps scrub starts 2024-03-24T12:08:16.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:16 smithi006 ceph-mon[41734]: purged_snaps scrub ok 2024-03-24T12:08:16.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:16 smithi006 ceph-mon[41734]: osd.5 [v2:172.21.15.90:6808/3585923902,v1:172.21.15.90:6809/3585923902] boot 2024-03-24T12:08:16.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:16 smithi006 ceph-mon[41734]: osdmap e38: 6 total, 6 up, 6 in 2024-03-24T12:08:16.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:16 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T12:08:16.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:16 smithi006 ceph-mon[41734]: pgmap v133: 1 pgs: 1 remapped; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-24T12:08:16.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:16 smithi006 ceph-mon[41734]: osdmap e39: 6 total, 6 up, 6 in 2024-03-24T12:08:17.705 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-24T12:08:17.705 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-24T12:08:17.705 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-03-24T12:08:17.705 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-03-24T12:08:17.705 INFO:teuthology.orchestra.run.smithi090.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0137382 s, 763 MB/s 2024-03-24T12:08:17.706 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-03-24T12:08:18.405 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi090:vg_nvme/lv_2 2024-03-24T12:08:18.659 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T12:08:18.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:18 smithi090 ceph-mon[33646]: osdmap e40: 6 total, 6 up, 6 in 2024-03-24T12:08:18.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:18 smithi090 ceph-mon[33646]: pgmap v136: 1 pgs: 1 remapped; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-24T12:08:18.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:18 smithi006 ceph-mon[30108]: osdmap e40: 6 total, 6 up, 6 in 2024-03-24T12:08:18.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:18 smithi006 ceph-mon[30108]: pgmap v136: 1 pgs: 1 remapped; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-24T12:08:18.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:18 smithi006 ceph-mon[41734]: osdmap e40: 6 total, 6 up, 6 in 2024-03-24T12:08:18.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:18 smithi006 ceph-mon[41734]: pgmap v136: 1 pgs: 1 remapped; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-24T12:08:19.617 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:19 smithi090 ceph-mon[33646]: osdmap e41: 6 total, 6 up, 6 in 2024-03-24T12:08:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:19 smithi006 ceph-mon[30108]: osdmap e41: 6 total, 6 up, 6 in 2024-03-24T12:08:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:19 smithi006 ceph-mon[41734]: osdmap e41: 6 total, 6 up, 6 in 2024-03-24T12:08:20.519 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:20 smithi090 ceph-mon[33646]: osdmap e42: 6 total, 6 up, 6 in 2024-03-24T12:08:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:20 smithi090 ceph-mon[33646]: pgmap v139: 1 pgs: 1 remapped; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-24T12:08:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:20 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:20 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:20 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:08:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:20 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:08:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:20 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:20 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:20 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:08:20.520 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:20 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:20.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.618+0000 7f1dbe7fe700 1 -- 172.21.15.90:0/1452728933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1db81027e0 msgr2=0x7f1db8109090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:08:20.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.618+0000 7f1dbe7fe700 1 --2- 172.21.15.90:0/1452728933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1db81027e0 0x7f1db8109090 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1db000b0d0 tx=0x7f1db000b3e0 comp rx=0 tx=0).stop 2024-03-24T12:08:20.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.618+0000 7f1dbe7fe700 1 -- 172.21.15.90:0/1452728933 shutdown_connections 2024-03-24T12:08:20.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.618+0000 7f1dbe7fe700 1 --2- 172.21.15.90:0/1452728933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1db81027e0 0x7f1db8109090 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:08:20.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.618+0000 7f1dbe7fe700 1 --2- 172.21.15.90:0/1452728933 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1db8100c40 0x7f1db8101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:08:20.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.618+0000 7f1dbe7fe700 1 --2- 172.21.15.90:0/1452728933 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1db8101e40 0x7f1db81022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:08:20.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.618+0000 7f1dbe7fe700 1 -- 172.21.15.90:0/1452728933 >> 172.21.15.90:0/1452728933 conn(0x7f1db80fc200 msgr2=0x7f1db80fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:08:20.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.618+0000 7f1dbe7fe700 1 -- 172.21.15.90:0/1452728933 shutdown_connections 2024-03-24T12:08:20.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.618+0000 7f1dbe7fe700 1 -- 172.21.15.90:0/1452728933 wait complete. 2024-03-24T12:08:20.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.619+0000 7f1dbe7fe700 1 Processor -- start 2024-03-24T12:08:20.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.619+0000 7f1dbe7fe700 1 -- start start 2024-03-24T12:08:20.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.619+0000 7f1dbe7fe700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1db8100c40 0x7f1db8197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:08:20.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.619+0000 7f1dbe7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1db8101e40 0x7f1db8197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:08:20.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.619+0000 7f1dbe7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1db81027e0 0x7f1db819e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:08:20.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.619+0000 7f1dbe7fe700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db819efc0 con 0x7f1db8101e40 2024-03-24T12:08:20.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.619+0000 7f1dbe7fe700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db819f100 con 0x7f1db81027e0 2024-03-24T12:08:20.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.619+0000 7f1dbe7fe700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db819f290 con 0x7f1db8100c40 2024-03-24T12:08:20.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.619+0000 7f1dbcd9b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1db81027e0 0x7f1db819e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:08:20.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.619+0000 7f1dbcd9b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1db81027e0 0x7f1db819e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38972/0 (socket says 172.21.15.90:38972) 2024-03-24T12:08:20.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.619+0000 7f1dbcd9b700 1 -- 172.21.15.90:0/2083705953 learned_addr learned my addr 172.21.15.90:0/2083705953 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-24T12:08:20.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.619+0000 7f1db7fff700 1 --2- 172.21.15.90:0/2083705953 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1db8100c40 0x7f1db8197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:08:20.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.619+0000 7f1db77fe700 1 --2- 172.21.15.90:0/2083705953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1db8101e40 0x7f1db8197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:08:20.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.619+0000 7f1dbcd9b700 1 -- 172.21.15.90:0/2083705953 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1db8100c40 msgr2=0x7f1db8197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:08:20.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.619+0000 7f1dbcd9b700 1 --2- 172.21.15.90:0/2083705953 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1db8100c40 0x7f1db8197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:08:20.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.619+0000 7f1dbcd9b700 1 -- 172.21.15.90:0/2083705953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1db8101e40 msgr2=0x7f1db8197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:08:20.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.619+0000 7f1dbcd9b700 1 --2- 172.21.15.90:0/2083705953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1db8101e40 0x7f1db8197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:08:20.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.620+0000 7f1dbcd9b700 1 -- 172.21.15.90:0/2083705953 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db0009d00 con 0x7f1db81027e0 2024-03-24T12:08:20.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.620+0000 7f1db77fe700 1 --2- 172.21.15.90:0/2083705953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1db8101e40 0x7f1db8197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T12:08:20.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.620+0000 7f1dbcd9b700 1 --2- 172.21.15.90:0/2083705953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1db81027e0 0x7f1db819e9f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1db0007930 tx=0x7f1db0009180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:08:20.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.620+0000 7f1db57fa700 1 -- 172.21.15.90:0/2083705953 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db000e040 con 0x7f1db81027e0 2024-03-24T12:08:20.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.620+0000 7f1db57fa700 1 -- 172.21.15.90:0/2083705953 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1db0009490 con 0x7f1db81027e0 2024-03-24T12:08:20.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.620+0000 7f1dbe7fe700 1 -- 172.21.15.90:0/2083705953 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db819f510 con 0x7f1db81027e0 2024-03-24T12:08:20.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.620+0000 7f1db57fa700 1 -- 172.21.15.90:0/2083705953 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db001b7b0 con 0x7f1db81027e0 2024-03-24T12:08:20.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.620+0000 7f1dbe7fe700 1 -- 172.21.15.90:0/2083705953 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db819f9a0 con 0x7f1db81027e0 2024-03-24T12:08:20.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.621+0000 7f1dbe7fe700 1 -- 172.21.15.90:0/2083705953 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db8108a20 con 0x7f1db81027e0 2024-03-24T12:08:20.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.621+0000 7f1db57fa700 1 -- 172.21.15.90:0/2083705953 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 15) v1 ==== 91079+0+0 (secure 0 0 0) 0x7f1db0024400 con 0x7f1db81027e0 2024-03-24T12:08:20.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.622+0000 7f1db57fa700 1 --2- 172.21.15.90:0/2083705953 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f1da006d570 0x7f1da006fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:08:20.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.622+0000 7f1db57fa700 1 -- 172.21.15.90:0/2083705953 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4423+0+0 (secure 0 0 0) 0x7f1db008dbc0 con 0x7f1db81027e0 2024-03-24T12:08:20.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.622+0000 7f1db7fff700 1 --2- 172.21.15.90:0/2083705953 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f1da006d570 0x7f1da006fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:08:20.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.622+0000 7f1db7fff700 1 --2- 172.21.15.90:0/2083705953 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f1da006d570 0x7f1da006fa30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1da8006fd0 tx=0x7f1da8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:08:20.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.626+0000 7f1db57fa700 1 -- 172.21.15.90:0/2083705953 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1db0057f50 con 0x7f1db81027e0 2024-03-24T12:08:20.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:20 smithi006 ceph-mon[30108]: osdmap e42: 6 total, 6 up, 6 in 2024-03-24T12:08:20.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:20 smithi006 ceph-mon[30108]: pgmap v139: 1 pgs: 1 remapped; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-24T12:08:20.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:20 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:20.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:20 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:20.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:20 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:08:20.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:20 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:08:20.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:20 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:20.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:20 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:20.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:20 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:08:20.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:20 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:20.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:20 smithi006 ceph-mon[41734]: osdmap e42: 6 total, 6 up, 6 in 2024-03-24T12:08:20.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:20 smithi006 ceph-mon[41734]: pgmap v139: 1 pgs: 1 remapped; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-24T12:08:20.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:20 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:20.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:20 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:20.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:20 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:08:20.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:20 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:08:20.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:20 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:20.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:20 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:20.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:20 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:08:20.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:20 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:20.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:20.824+0000 7f1dbe7fe700 1 -- 172.21.15.90:0/2083705953 --> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f1db80620b0 con 0x7f1da006d570 2024-03-24T12:08:21.545 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:21 smithi090 ceph-mon[33646]: Detected new or changed devices on smithi090 2024-03-24T12:08:21.545 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:21 smithi090 ceph-mon[33646]: Adjusting osd_memory_target on smithi090 to 8544M 2024-03-24T12:08:21.545 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:08:21.546 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:21.546 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:08:21.546 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:21.546 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:21 smithi090 ceph-mon[33646]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:08:21.546 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:08:21.546 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:08:21.546 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:21 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:21.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:21 smithi006 ceph-mon[30108]: Detected new or changed devices on smithi090 2024-03-24T12:08:21.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:21 smithi006 ceph-mon[30108]: Adjusting osd_memory_target on smithi090 to 8544M 2024-03-24T12:08:21.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:21 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:08:21.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:21 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:21.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:21 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:08:21.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:21 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:21.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:21 smithi006 ceph-mon[30108]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:08:21.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:21 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:08:21.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:21 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:08:21.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:21 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:21.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:21 smithi006 ceph-mon[41734]: Detected new or changed devices on smithi090 2024-03-24T12:08:21.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:21 smithi006 ceph-mon[41734]: Adjusting osd_memory_target on smithi090 to 8544M 2024-03-24T12:08:21.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:21 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:08:21.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:21 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:21.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:21 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:08:21.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:21 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:21.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:21 smithi006 ceph-mon[41734]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:08:21.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:21 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:08:21.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:21 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:08:21.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:21 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:22.592 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:22 smithi090 ceph-mon[33646]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-24T12:08:22.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:22 smithi006 ceph-mon[30108]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-24T12:08:22.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:22 smithi006 ceph-mon[41734]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-24T12:08:23.796 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.90:0/2303555356' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43226b7c-1700-47e4-8d1b-7f765e8defda"}]: dispatch 2024-03-24T12:08:23.796 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43226b7c-1700-47e4-8d1b-7f765e8defda"}]: dispatch 2024-03-24T12:08:23.796 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43226b7c-1700-47e4-8d1b-7f765e8defda"}]': finished 2024-03-24T12:08:23.796 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:23 smithi090 ceph-mon[33646]: osdmap e43: 7 total, 6 up, 7 in 2024-03-24T12:08:23.796 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:23 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T12:08:23.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.90:0/2303555356' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43226b7c-1700-47e4-8d1b-7f765e8defda"}]: dispatch 2024-03-24T12:08:23.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43226b7c-1700-47e4-8d1b-7f765e8defda"}]: dispatch 2024-03-24T12:08:23.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43226b7c-1700-47e4-8d1b-7f765e8defda"}]': finished 2024-03-24T12:08:23.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:23 smithi006 ceph-mon[30108]: osdmap e43: 7 total, 6 up, 7 in 2024-03-24T12:08:23.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:23 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T12:08:23.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.90:0/2303555356' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43226b7c-1700-47e4-8d1b-7f765e8defda"}]: dispatch 2024-03-24T12:08:23.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43226b7c-1700-47e4-8d1b-7f765e8defda"}]: dispatch 2024-03-24T12:08:23.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43226b7c-1700-47e4-8d1b-7f765e8defda"}]': finished 2024-03-24T12:08:23.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:23 smithi006 ceph-mon[41734]: osdmap e43: 7 total, 6 up, 7 in 2024-03-24T12:08:23.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:23 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T12:08:24.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:24 smithi090 ceph-mon[33646]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-24T12:08:24.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.90:0/762143855' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:08:24.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:24 smithi006 ceph-mon[30108]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-24T12:08:24.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.90:0/762143855' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:08:24.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:24 smithi006 ceph-mon[41734]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-24T12:08:24.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.90:0/762143855' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:08:26.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:25 smithi006 ceph-mon[30108]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-24T12:08:26.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:25 smithi006 ceph-mon[41734]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-24T12:08:26.294 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:25 smithi090 ceph-mon[33646]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-24T12:08:28.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:27 smithi090 ceph-mon[33646]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-24T12:08:28.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:27 smithi006 ceph-mon[30108]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-24T12:08:28.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:27 smithi006 ceph-mon[41734]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-24T12:08:30.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:29 smithi090 ceph-mon[33646]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-24T12:08:30.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:29 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-24T12:08:30.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:29 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:30.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:29 smithi090 ceph-mon[33646]: Deploying daemon osd.6 on smithi090 2024-03-24T12:08:30.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:29 smithi006 ceph-mon[30108]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-24T12:08:30.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:29 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-24T12:08:30.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:29 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:30.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:29 smithi006 ceph-mon[30108]: Deploying daemon osd.6 on smithi090 2024-03-24T12:08:30.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:29 smithi006 ceph-mon[41734]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-24T12:08:30.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:29 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-24T12:08:30.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:29 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:30.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:29 smithi006 ceph-mon[41734]: Deploying daemon osd.6 on smithi090 2024-03-24T12:08:32.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:31 smithi090 ceph-mon[33646]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-24T12:08:32.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:31 smithi006 ceph-mon[30108]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-24T12:08:32.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:31 smithi006 ceph-mon[41734]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-24T12:08:34.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:33 smithi090 ceph-mon[33646]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-24T12:08:34.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:33 smithi006 ceph-mon[30108]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-24T12:08:34.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:33 smithi006 ceph-mon[41734]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-24T12:08:36.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:36 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:36.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:36 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:36.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:36 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:08:36.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:36 smithi090 ceph-mon[33646]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-24T12:08:36.913 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:36 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:36.913 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:36 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:36.913 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:36 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:08:36.913 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:36 smithi006 ceph-mon[30108]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-24T12:08:36.913 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:36 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:36.914 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:36 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:36.914 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:36 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:08:36.914 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:36 smithi006 ceph-mon[41734]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-24T12:08:38.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:37 smithi006 ceph-mon[30108]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-24T12:08:38.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:37 smithi006 ceph-mon[41734]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-24T12:08:38.244 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:37 smithi090 ceph-mon[33646]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-24T12:08:38.937 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 6 on host 'smithi090' 2024-03-24T12:08:38.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:38.937+0000 7f1db57fa700 1 -- 172.21.15.90:0/2083705953 <== mgr.14152 v2:172.21.15.6:6800/3847865179 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1db80620b0 con 0x7f1da006d570 2024-03-24T12:08:38.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:38.939+0000 7f1dbe7fe700 1 -- 172.21.15.90:0/2083705953 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f1da006d570 msgr2=0x7f1da006fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:08:38.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:38.939+0000 7f1dbe7fe700 1 --2- 172.21.15.90:0/2083705953 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f1da006d570 0x7f1da006fa30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1da8006fd0 tx=0x7f1da8008040 comp rx=0 tx=0).stop 2024-03-24T12:08:38.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:38.939+0000 7f1dbe7fe700 1 -- 172.21.15.90:0/2083705953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1db81027e0 msgr2=0x7f1db819e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:08:38.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:38.939+0000 7f1dbe7fe700 1 --2- 172.21.15.90:0/2083705953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1db81027e0 0x7f1db819e9f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1db0007930 tx=0x7f1db0009180 comp rx=0 tx=0).stop 2024-03-24T12:08:38.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:38.939+0000 7f1dbe7fe700 1 -- 172.21.15.90:0/2083705953 shutdown_connections 2024-03-24T12:08:38.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:38.939+0000 7f1dbe7fe700 1 --2- 172.21.15.90:0/2083705953 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f1da006d570 0x7f1da006fa30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:08:38.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:38.939+0000 7f1dbe7fe700 1 --2- 172.21.15.90:0/2083705953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1db81027e0 0x7f1db819e9f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:08:38.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:38.939+0000 7f1dbe7fe700 1 --2- 172.21.15.90:0/2083705953 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1db8100c40 0x7f1db8197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:08:38.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:38.939+0000 7f1dbe7fe700 1 --2- 172.21.15.90:0/2083705953 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1db8101e40 0x7f1db8197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:08:38.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:38.939+0000 7f1dbe7fe700 1 -- 172.21.15.90:0/2083705953 >> 172.21.15.90:0/2083705953 conn(0x7f1db80fc200 msgr2=0x7f1db806ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:08:38.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:38.939+0000 7f1dbe7fe700 1 -- 172.21.15.90:0/2083705953 shutdown_connections 2024-03-24T12:08:38.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:38.940+0000 7f1dbe7fe700 1 -- 172.21.15.90:0/2083705953 wait complete. 2024-03-24T12:08:39.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:38 smithi090 ceph-mon[33646]: from='osd.6 [v2:172.21.15.90:6816/713247979,v1:172.21.15.90:6817/713247979]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-24T12:08:39.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:38 smithi090 ceph-mon[33646]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-24T12:08:39.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:38 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:39.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:38 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:39.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:38 smithi006 ceph-mon[30108]: from='osd.6 [v2:172.21.15.90:6816/713247979,v1:172.21.15.90:6817/713247979]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-24T12:08:39.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:38 smithi006 ceph-mon[30108]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-24T12:08:39.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:38 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:39.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:38 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:39.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:38 smithi006 ceph-mon[41734]: from='osd.6 [v2:172.21.15.90:6816/713247979,v1:172.21.15.90:6817/713247979]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-24T12:08:39.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:38 smithi006 ceph-mon[41734]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-24T12:08:39.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:38 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:39.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:38 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:39.562 DEBUG:teuthology.orchestra.run.smithi090:osd.6> sudo journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.6.service 2024-03-24T12:08:39.564 INFO:tasks.cephadm:Deploying osd.7 on smithi090 with /dev/vg_nvme/lv_1... 2024-03-24T12:08:39.564 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-24T12:08:39.850 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T12:08:40.214 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:39 smithi090 ceph-mon[33646]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-24T12:08:40.214 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:39 smithi090 ceph-mon[33646]: osdmap e44: 7 total, 6 up, 7 in 2024-03-24T12:08:40.215 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:39 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T12:08:40.215 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:39 smithi090 ceph-mon[33646]: from='osd.6 [v2:172.21.15.90:6816/713247979,v1:172.21.15.90:6817/713247979]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:08:40.215 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:39 smithi090 ceph-mon[33646]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:08:40.215 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:39 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:40.215 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:39 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:40.215 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:39 smithi090 ceph-mon[33646]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-24T12:08:40.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:39 smithi006 ceph-mon[30108]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-24T12:08:40.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:39 smithi006 ceph-mon[30108]: osdmap e44: 7 total, 6 up, 7 in 2024-03-24T12:08:40.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:39 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T12:08:40.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:39 smithi006 ceph-mon[30108]: from='osd.6 [v2:172.21.15.90:6816/713247979,v1:172.21.15.90:6817/713247979]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:08:40.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:39 smithi006 ceph-mon[30108]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:08:40.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:39 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:40.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:39 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:40.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:39 smithi006 ceph-mon[30108]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-24T12:08:40.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:39 smithi006 ceph-mon[41734]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-24T12:08:40.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:39 smithi006 ceph-mon[41734]: osdmap e44: 7 total, 6 up, 7 in 2024-03-24T12:08:40.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:39 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T12:08:40.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:39 smithi006 ceph-mon[41734]: from='osd.6 [v2:172.21.15.90:6816/713247979,v1:172.21.15.90:6817/713247979]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:08:40.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:39 smithi006 ceph-mon[41734]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:08:40.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:39 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:40.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:39 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:40.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:39 smithi006 ceph-mon[41734]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-24T12:08:40.650 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:08:40 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:08:40.393+0000 7f1d4a77f700 -1 osd.6 0 waiting for initial osdmap 2024-03-24T12:08:40.651 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:08:40 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:08:40.399+0000 7f1d4256a700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T12:08:41.257 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:40 smithi090 ceph-mon[33646]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-03-24T12:08:41.258 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:40 smithi090 ceph-mon[33646]: osdmap e45: 7 total, 6 up, 7 in 2024-03-24T12:08:41.258 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:40 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T12:08:41.258 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:40 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T12:08:41.258 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:40 smithi090 ceph-mon[33646]: from='osd.6 ' entity='osd.6' 2024-03-24T12:08:41.258 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:40 smithi090 ceph-mon[33646]: osd.6 [v2:172.21.15.90:6816/713247979,v1:172.21.15.90:6817/713247979] boot 2024-03-24T12:08:41.258 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:40 smithi090 ceph-mon[33646]: osdmap e46: 7 total, 7 up, 7 in 2024-03-24T12:08:41.258 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:40 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T12:08:41.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:40 smithi006 ceph-mon[41734]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-03-24T12:08:41.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:40 smithi006 ceph-mon[41734]: osdmap e45: 7 total, 6 up, 7 in 2024-03-24T12:08:41.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:40 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T12:08:41.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:40 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T12:08:41.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:40 smithi006 ceph-mon[41734]: from='osd.6 ' entity='osd.6' 2024-03-24T12:08:41.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:40 smithi006 ceph-mon[41734]: osd.6 [v2:172.21.15.90:6816/713247979,v1:172.21.15.90:6817/713247979] boot 2024-03-24T12:08:41.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:40 smithi006 ceph-mon[41734]: osdmap e46: 7 total, 7 up, 7 in 2024-03-24T12:08:41.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:40 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T12:08:41.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:40 smithi006 ceph-mon[30108]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-03-24T12:08:41.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:40 smithi006 ceph-mon[30108]: osdmap e45: 7 total, 6 up, 7 in 2024-03-24T12:08:41.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:40 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T12:08:41.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:40 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T12:08:41.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:40 smithi006 ceph-mon[30108]: from='osd.6 ' entity='osd.6' 2024-03-24T12:08:41.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:40 smithi006 ceph-mon[30108]: osd.6 [v2:172.21.15.90:6816/713247979,v1:172.21.15.90:6817/713247979] boot 2024-03-24T12:08:41.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:40 smithi006 ceph-mon[30108]: osdmap e46: 7 total, 7 up, 7 in 2024-03-24T12:08:41.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:40 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T12:08:41.844 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-24T12:08:41.844 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-24T12:08:41.844 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-03-24T12:08:41.844 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-03-24T12:08:41.844 INFO:teuthology.orchestra.run.smithi090.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0157011 s, 668 MB/s 2024-03-24T12:08:41.844 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-03-24T12:08:42.671 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:42 smithi090 ceph-mon[33646]: purged_snaps scrub starts 2024-03-24T12:08:42.671 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:42 smithi090 ceph-mon[33646]: purged_snaps scrub ok 2024-03-24T12:08:42.671 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:42 smithi090 ceph-mon[33646]: osdmap e47: 7 total, 7 up, 7 in 2024-03-24T12:08:42.671 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:42 smithi090 ceph-mon[33646]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:08:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:42 smithi006 ceph-mon[30108]: purged_snaps scrub starts 2024-03-24T12:08:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:42 smithi006 ceph-mon[30108]: purged_snaps scrub ok 2024-03-24T12:08:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:42 smithi006 ceph-mon[30108]: osdmap e47: 7 total, 7 up, 7 in 2024-03-24T12:08:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:42 smithi006 ceph-mon[30108]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:08:42.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:42 smithi006 ceph-mon[41734]: purged_snaps scrub starts 2024-03-24T12:08:42.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:42 smithi006 ceph-mon[41734]: purged_snaps scrub ok 2024-03-24T12:08:42.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:42 smithi006 ceph-mon[41734]: osdmap e47: 7 total, 7 up, 7 in 2024-03-24T12:08:42.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:42 smithi006 ceph-mon[41734]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:08:43.393 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi090:vg_nvme/lv_1 2024-03-24T12:08:43.623 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:43 smithi090 ceph-mon[33646]: osdmap e48: 7 total, 7 up, 7 in 2024-03-24T12:08:43.691 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T12:08:43.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:43 smithi006 ceph-mon[30108]: osdmap e48: 7 total, 7 up, 7 in 2024-03-24T12:08:43.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:43 smithi006 ceph-mon[41734]: osdmap e48: 7 total, 7 up, 7 in 2024-03-24T12:08:44.726 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:44 smithi090 ceph-mon[33646]: osdmap e49: 7 total, 7 up, 7 in 2024-03-24T12:08:44.726 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:44 smithi090 ceph-mon[33646]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:08:44.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:44 smithi006 ceph-mon[30108]: osdmap e49: 7 total, 7 up, 7 in 2024-03-24T12:08:44.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:44 smithi006 ceph-mon[30108]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:08:44.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:44 smithi006 ceph-mon[41734]: osdmap e49: 7 total, 7 up, 7 in 2024-03-24T12:08:44.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:44 smithi006 ceph-mon[41734]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:08:45.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:45 smithi090 ceph-mon[33646]: osdmap e50: 7 total, 7 up, 7 in 2024-03-24T12:08:45.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:45 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:45.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:45 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:45.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:45 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:08:45.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:45 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:08:45.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:45 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:08:45.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:45 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:45.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:45 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:45.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:45 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:08:45.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:45 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:45.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:45 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:08:45.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:45 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:45.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:45 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:08:45.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:45 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:45.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.905+0000 7fed1b85e700 1 -- 172.21.15.90:0/3450596757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed14103880 msgr2=0x7fed1410a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:08:45.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.905+0000 7fed1b85e700 1 --2- 172.21.15.90:0/3450596757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed14103880 0x7fed1410a130 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00b0d0 tx=0x7fed0c00b3e0 comp rx=0 tx=0).stop 2024-03-24T12:08:45.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.905+0000 7fed1b85e700 1 -- 172.21.15.90:0/3450596757 shutdown_connections 2024-03-24T12:08:45.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.905+0000 7fed1b85e700 1 --2- 172.21.15.90:0/3450596757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed14103880 0x7fed1410a130 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:08:45.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.905+0000 7fed1b85e700 1 --2- 172.21.15.90:0/3450596757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed14101ce0 0x7fed14102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:08:45.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.905+0000 7fed1b85e700 1 --2- 172.21.15.90:0/3450596757 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fed14102ee0 0x7fed14103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:08:45.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.905+0000 7fed1b85e700 1 -- 172.21.15.90:0/3450596757 >> 172.21.15.90:0/3450596757 conn(0x7fed140fd2c0 msgr2=0x7fed140ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:08:45.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.905+0000 7fed1b85e700 1 -- 172.21.15.90:0/3450596757 shutdown_connections 2024-03-24T12:08:45.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.905+0000 7fed1b85e700 1 -- 172.21.15.90:0/3450596757 wait complete. 2024-03-24T12:08:45.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.906+0000 7fed1b85e700 1 Processor -- start 2024-03-24T12:08:45.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.906+0000 7fed1b85e700 1 -- start start 2024-03-24T12:08:45.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.906+0000 7fed1b85e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed14101ce0 0x7fed140759c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:08:45.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.906+0000 7fed1b85e700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fed14102ee0 0x7fed14074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:08:45.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.906+0000 7fed1b85e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed14103880 0x7fed140745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:08:45.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.906+0000 7fed1b85e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed14074d40 con 0x7fed14103880 2024-03-24T12:08:45.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.906+0000 7fed1b85e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed14074eb0 con 0x7fed14101ce0 2024-03-24T12:08:45.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.906+0000 7fed1b85e700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed141a1330 con 0x7fed14102ee0 2024-03-24T12:08:45.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.906+0000 7fed195fa700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed14101ce0 0x7fed140759c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:08:45.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.906+0000 7fed195fa700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed14101ce0 0x7fed140759c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46772/0 (socket says 172.21.15.90:46772) 2024-03-24T12:08:45.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.906+0000 7fed195fa700 1 -- 172.21.15.90:0/2300611716 learned_addr learned my addr 172.21.15.90:0/2300611716 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-24T12:08:45.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.906+0000 7fed19dfb700 1 --2- 172.21.15.90:0/2300611716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed14103880 0x7fed140745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:08:45.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.906+0000 7fed18df9700 1 --2- 172.21.15.90:0/2300611716 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fed14102ee0 0x7fed14074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:08:45.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.906+0000 7fed195fa700 1 -- 172.21.15.90:0/2300611716 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fed14102ee0 msgr2=0x7fed14074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:08:45.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.906+0000 7fed195fa700 1 --2- 172.21.15.90:0/2300611716 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fed14102ee0 0x7fed14074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:08:45.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.906+0000 7fed195fa700 1 -- 172.21.15.90:0/2300611716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed14103880 msgr2=0x7fed140745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:08:45.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.906+0000 7fed195fa700 1 --2- 172.21.15.90:0/2300611716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed14103880 0x7fed140745a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:08:45.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.906+0000 7fed195fa700 1 -- 172.21.15.90:0/2300611716 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed0c009d00 con 0x7fed14101ce0 2024-03-24T12:08:45.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.907+0000 7fed195fa700 1 --2- 172.21.15.90:0/2300611716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed14101ce0 0x7fed140759c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fed1000d980 tx=0x7fed1000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:08:45.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.907+0000 7fed0a7fc700 1 -- 172.21.15.90:0/2300611716 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10004020 con 0x7fed14101ce0 2024-03-24T12:08:45.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.907+0000 7fed1b85e700 1 -- 172.21.15.90:0/2300611716 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed141a15c0 con 0x7fed14101ce0 2024-03-24T12:08:45.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.907+0000 7fed0a7fc700 1 -- 172.21.15.90:0/2300611716 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fed1000f690 con 0x7fed14101ce0 2024-03-24T12:08:45.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.907+0000 7fed0a7fc700 1 -- 172.21.15.90:0/2300611716 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10003950 con 0x7fed14101ce0 2024-03-24T12:08:45.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.907+0000 7fed1b85e700 1 -- 172.21.15.90:0/2300611716 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed141a1b10 con 0x7fed14101ce0 2024-03-24T12:08:45.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.908+0000 7fed1b85e700 1 -- 172.21.15.90:0/2300611716 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed141088c0 con 0x7fed14101ce0 2024-03-24T12:08:45.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.908+0000 7fed0a7fc700 1 -- 172.21.15.90:0/2300611716 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 15) v1 ==== 91079+0+0 (secure 0 0 0) 0x7fed10004180 con 0x7fed14101ce0 2024-03-24T12:08:45.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.909+0000 7fed0a7fc700 1 --2- 172.21.15.90:0/2300611716 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fed0006d570 0x7fed0006fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:08:45.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.909+0000 7fed0a7fc700 1 -- 172.21.15.90:0/2300611716 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4855+0+0 (secure 0 0 0) 0x7fed1008c290 con 0x7fed14101ce0 2024-03-24T12:08:45.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.909+0000 7fed18df9700 1 --2- 172.21.15.90:0/2300611716 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fed0006d570 0x7fed0006fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:08:45.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.910+0000 7fed18df9700 1 --2- 172.21.15.90:0/2300611716 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fed0006d570 0x7fed0006fa30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fed04005950 tx=0x7fed0400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:08:45.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:45.913+0000 7fed0a7fc700 1 -- 172.21.15.90:0/2300611716 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fed10056470 con 0x7fed14101ce0 2024-03-24T12:08:45.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[30108]: osdmap e50: 7 total, 7 up, 7 in 2024-03-24T12:08:45.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:45.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:45.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:08:45.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:08:45.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:08:45.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:45.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:45.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:08:45.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:45.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:08:45.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:45.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:08:45.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:45.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[41734]: osdmap e50: 7 total, 7 up, 7 in 2024-03-24T12:08:45.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:45.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:45.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:08:45.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:08:45.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:08:45.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:45.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:45.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:08:45.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:45.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:08:45.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:45.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:08:45.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:45 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:08:46.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:08:46.114+0000 7fed1b85e700 1 -- 172.21.15.90:0/2300611716 --> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fed1410cd10 con 0x7fed0006d570 2024-03-24T12:08:46.753 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:46 smithi090 ceph-mon[33646]: Detected new or changed devices on smithi090 2024-03-24T12:08:46.753 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:46 smithi090 ceph-mon[33646]: Adjusting osd_memory_target on smithi090 to 5696M 2024-03-24T12:08:46.753 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:46 smithi090 ceph-mon[33646]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:08:46.753 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:46 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:08:46.753 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:46 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:08:46.753 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:46 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:46.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:46 smithi006 ceph-mon[30108]: Detected new or changed devices on smithi090 2024-03-24T12:08:46.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:46 smithi006 ceph-mon[30108]: Adjusting osd_memory_target on smithi090 to 5696M 2024-03-24T12:08:46.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:46 smithi006 ceph-mon[30108]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:08:46.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:46 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:08:46.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:46 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:08:46.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:46 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:46.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:46 smithi006 ceph-mon[41734]: Detected new or changed devices on smithi090 2024-03-24T12:08:46.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:46 smithi006 ceph-mon[41734]: Adjusting osd_memory_target on smithi090 to 5696M 2024-03-24T12:08:46.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:46 smithi006 ceph-mon[41734]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:08:46.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:46 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-24T12:08:46.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:46 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-24T12:08:46.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:46 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:47.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:47 smithi090 ceph-mon[33646]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:08:47.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:47 smithi006 ceph-mon[41734]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:08:47.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:47 smithi006 ceph-mon[30108]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:08:48.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:48 smithi006 ceph-mon[30108]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-24T12:08:48.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:48 smithi006 ceph-mon[41734]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-24T12:08:48.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:48 smithi090 ceph-mon[33646]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-24T12:08:49.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:49 smithi090 ceph-mon[33646]: from='client.? 172.21.15.90:0/1123567547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa0def8c-eda9-441c-8ee9-e0021c064bf7"}]: dispatch 2024-03-24T12:08:49.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:49 smithi090 ceph-mon[33646]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa0def8c-eda9-441c-8ee9-e0021c064bf7"}]: dispatch 2024-03-24T12:08:49.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:49 smithi090 ceph-mon[33646]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa0def8c-eda9-441c-8ee9-e0021c064bf7"}]': finished 2024-03-24T12:08:49.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:49 smithi090 ceph-mon[33646]: osdmap e51: 8 total, 7 up, 8 in 2024-03-24T12:08:49.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:49 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T12:08:49.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:49 smithi090 ceph-mon[33646]: from='client.? 172.21.15.90:0/757704393' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:08:49.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:49 smithi006 ceph-mon[41734]: from='client.? 172.21.15.90:0/1123567547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa0def8c-eda9-441c-8ee9-e0021c064bf7"}]: dispatch 2024-03-24T12:08:49.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:49 smithi006 ceph-mon[41734]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa0def8c-eda9-441c-8ee9-e0021c064bf7"}]: dispatch 2024-03-24T12:08:49.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:49 smithi006 ceph-mon[41734]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa0def8c-eda9-441c-8ee9-e0021c064bf7"}]': finished 2024-03-24T12:08:49.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:49 smithi006 ceph-mon[41734]: osdmap e51: 8 total, 7 up, 8 in 2024-03-24T12:08:49.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:49 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T12:08:49.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:49 smithi006 ceph-mon[41734]: from='client.? 172.21.15.90:0/757704393' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:08:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:49 smithi006 ceph-mon[30108]: from='client.? 172.21.15.90:0/1123567547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa0def8c-eda9-441c-8ee9-e0021c064bf7"}]: dispatch 2024-03-24T12:08:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:49 smithi006 ceph-mon[30108]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa0def8c-eda9-441c-8ee9-e0021c064bf7"}]: dispatch 2024-03-24T12:08:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:49 smithi006 ceph-mon[30108]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa0def8c-eda9-441c-8ee9-e0021c064bf7"}]': finished 2024-03-24T12:08:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:49 smithi006 ceph-mon[30108]: osdmap e51: 8 total, 7 up, 8 in 2024-03-24T12:08:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:49 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T12:08:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:49 smithi006 ceph-mon[30108]: from='client.? 172.21.15.90:0/757704393' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-24T12:08:50.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:50 smithi090 ceph-mon[33646]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-03-24T12:08:50.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:50 smithi006 ceph-mon[30108]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-03-24T12:08:50.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:50 smithi006 ceph-mon[41734]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-03-24T12:08:52.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:51 smithi006 ceph-mon[30108]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-24T12:08:52.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:51 smithi006 ceph-mon[41734]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-24T12:08:52.288 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:51 smithi090 ceph-mon[33646]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-24T12:08:54.176 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:53 smithi090 ceph-mon[33646]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-03-24T12:08:54.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:53 smithi006 ceph-mon[30108]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-03-24T12:08:54.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:53 smithi006 ceph-mon[41734]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-03-24T12:08:55.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:55 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-24T12:08:55.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:55 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:55.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:55 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-24T12:08:55.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:55 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:55.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:55 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-24T12:08:55.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:55 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:08:56.455 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:56 smithi090 ceph-mon[33646]: Deploying daemon osd.7 on smithi090 2024-03-24T12:08:56.455 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:56 smithi090 ceph-mon[33646]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-24T12:08:56.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:56 smithi006 ceph-mon[30108]: Deploying daemon osd.7 on smithi090 2024-03-24T12:08:56.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:56 smithi006 ceph-mon[30108]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-24T12:08:56.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:56 smithi006 ceph-mon[41734]: Deploying daemon osd.7 on smithi090 2024-03-24T12:08:56.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:56 smithi006 ceph-mon[41734]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-24T12:08:58.098 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:57 smithi090 ceph-mon[33646]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:08:58.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:57 smithi006 ceph-mon[30108]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:08:58.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:57 smithi006 ceph-mon[41734]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:09:00.108 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:08:59 smithi090 ceph-mon[33646]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:09:00.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:08:59 smithi006 ceph-mon[41734]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:09:00.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:08:59 smithi006 ceph-mon[30108]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:09:02.172 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:01 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:02.173 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:01 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:02.173 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:01 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:09:02.173 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:01 smithi090 ceph-mon[33646]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:09:02.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:01 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:02.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:01 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:02.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:01 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:09:02.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:01 smithi006 ceph-mon[30108]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:09:02.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:01 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:02.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:01 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:02.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:01 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:09:02.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:01 smithi006 ceph-mon[41734]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:09:04.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:03 smithi006 ceph-mon[30108]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:09:04.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:03 smithi006 ceph-mon[41734]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:09:04.304 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:03 smithi090 ceph-mon[33646]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:09:04.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:04.380+0000 7fed0a7fc700 1 -- 172.21.15.90:0/2300611716 <== mgr.14152 v2:172.21.15.6:6800/3847865179 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fed1410cd10 con 0x7fed0006d570 2024-03-24T12:09:04.380 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 7 on host 'smithi090' 2024-03-24T12:09:04.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:04.383+0000 7fed1b85e700 1 -- 172.21.15.90:0/2300611716 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fed0006d570 msgr2=0x7fed0006fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:04.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:04.383+0000 7fed1b85e700 1 --2- 172.21.15.90:0/2300611716 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fed0006d570 0x7fed0006fa30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fed04005950 tx=0x7fed0400b410 comp rx=0 tx=0).stop 2024-03-24T12:09:04.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:04.383+0000 7fed1b85e700 1 -- 172.21.15.90:0/2300611716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed14101ce0 msgr2=0x7fed140759c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:04.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:04.383+0000 7fed1b85e700 1 --2- 172.21.15.90:0/2300611716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed14101ce0 0x7fed140759c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fed1000d980 tx=0x7fed1000dc90 comp rx=0 tx=0).stop 2024-03-24T12:09:04.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:04.383+0000 7fed1b85e700 1 -- 172.21.15.90:0/2300611716 shutdown_connections 2024-03-24T12:09:04.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:04.383+0000 7fed1b85e700 1 --2- 172.21.15.90:0/2300611716 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fed0006d570 0x7fed0006fa30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:04.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:04.383+0000 7fed1b85e700 1 --2- 172.21.15.90:0/2300611716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed14103880 0x7fed140745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:04.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:04.383+0000 7fed1b85e700 1 --2- 172.21.15.90:0/2300611716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed14101ce0 0x7fed140759c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:04.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:04.383+0000 7fed1b85e700 1 --2- 172.21.15.90:0/2300611716 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fed14102ee0 0x7fed14074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:04.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:04.383+0000 7fed1b85e700 1 -- 172.21.15.90:0/2300611716 >> 172.21.15.90:0/2300611716 conn(0x7fed140fd2c0 msgr2=0x7fed140fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:04.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:04.383+0000 7fed1b85e700 1 -- 172.21.15.90:0/2300611716 shutdown_connections 2024-03-24T12:09:04.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:04.383+0000 7fed1b85e700 1 -- 172.21.15.90:0/2300611716 wait complete. 2024-03-24T12:09:04.971 DEBUG:teuthology.orchestra.run.smithi090:osd.7> sudo journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.7.service 2024-03-24T12:09:04.973 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-03-24T12:09:04.973 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-24T12:09:05.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:04 smithi006 ceph-mon[30108]: from='osd.7 [v2:172.21.15.90:6824/1629523727,v1:172.21.15.90:6825/1629523727]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T12:09:05.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:04 smithi006 ceph-mon[30108]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T12:09:05.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:04 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:05.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:04 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:05.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:04 smithi006 ceph-mon[41734]: from='osd.7 [v2:172.21.15.90:6824/1629523727,v1:172.21.15.90:6825/1629523727]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T12:09:05.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:04 smithi006 ceph-mon[41734]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T12:09:05.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:04 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:05.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:04 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:05.229 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:09:05.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:04 smithi090 ceph-mon[33646]: from='osd.7 [v2:172.21.15.90:6824/1629523727,v1:172.21.15.90:6825/1629523727]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T12:09:05.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:04 smithi090 ceph-mon[33646]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-24T12:09:05.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:04 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:05.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:04 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:06.188 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:05 smithi006 ceph-mon[30108]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-24T12:09:06.188 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:05 smithi006 ceph-mon[30108]: osdmap e52: 8 total, 7 up, 8 in 2024-03-24T12:09:06.188 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:05 smithi006 ceph-mon[30108]: from='osd.7 [v2:172.21.15.90:6824/1629523727,v1:172.21.15.90:6825/1629523727]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:09:06.188 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:05 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T12:09:06.188 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:05 smithi006 ceph-mon[30108]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:09:06.188 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:05 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:06.188 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:05 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:06.188 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:05 smithi006 ceph-mon[30108]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:09:06.189 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:05 smithi006 ceph-mon[41734]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-24T12:09:06.189 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:05 smithi006 ceph-mon[41734]: osdmap e52: 8 total, 7 up, 8 in 2024-03-24T12:09:06.189 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:05 smithi006 ceph-mon[41734]: from='osd.7 [v2:172.21.15.90:6824/1629523727,v1:172.21.15.90:6825/1629523727]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:09:06.189 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:05 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T12:09:06.189 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:05 smithi006 ceph-mon[41734]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:09:06.189 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:05 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:06.189 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:05 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:06.189 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:05 smithi006 ceph-mon[41734]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:09:06.385 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:05 smithi090 ceph-mon[33646]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-24T12:09:06.385 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:05 smithi090 ceph-mon[33646]: osdmap e52: 8 total, 7 up, 8 in 2024-03-24T12:09:06.385 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:05 smithi090 ceph-mon[33646]: from='osd.7 [v2:172.21.15.90:6824/1629523727,v1:172.21.15.90:6825/1629523727]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:09:06.386 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:05 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T12:09:06.386 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:05 smithi090 ceph-mon[33646]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-03-24T12:09:06.386 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:05 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:06.386 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:05 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:06.386 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:05 smithi090 ceph-mon[33646]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-24T12:09:06.386 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:09:05 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:09:05.969+0000 7fb22ed19700 -1 osd.7 0 waiting for initial osdmap 2024-03-24T12:09:06.386 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:09:05 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:09:05.974+0000 7fb22ab0c700 -1 osd.7 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-24T12:09:07.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:06 smithi006 ceph-mon[30108]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-03-24T12:09:07.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:06 smithi006 ceph-mon[30108]: osdmap e53: 8 total, 7 up, 8 in 2024-03-24T12:09:07.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:06 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T12:09:07.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:06 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T12:09:07.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:06 smithi006 ceph-mon[30108]: osd.7 [v2:172.21.15.90:6824/1629523727,v1:172.21.15.90:6825/1629523727] boot 2024-03-24T12:09:07.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:06 smithi006 ceph-mon[30108]: osdmap e54: 8 total, 8 up, 8 in 2024-03-24T12:09:07.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:06 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T12:09:07.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:06 smithi006 ceph-mon[41734]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-03-24T12:09:07.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:06 smithi006 ceph-mon[41734]: osdmap e53: 8 total, 7 up, 8 in 2024-03-24T12:09:07.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:06 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T12:09:07.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:06 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T12:09:07.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:06 smithi006 ceph-mon[41734]: osd.7 [v2:172.21.15.90:6824/1629523727,v1:172.21.15.90:6825/1629523727] boot 2024-03-24T12:09:07.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:06 smithi006 ceph-mon[41734]: osdmap e54: 8 total, 8 up, 8 in 2024-03-24T12:09:07.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:06 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T12:09:07.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.250+0000 7fe271781700 1 -- 172.21.15.6:0/14996378 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe26c100b90 msgr2=0x7fe26c100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:07.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.250+0000 7fe271781700 1 --2- 172.21.15.6:0/14996378 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe26c100b90 0x7fe26c100fb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe25c00b0a0 tx=0x7fe25c00b3b0 comp rx=0 tx=0).stop 2024-03-24T12:09:07.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.250+0000 7fe271781700 1 -- 172.21.15.6:0/14996378 shutdown_connections 2024-03-24T12:09:07.249 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.250+0000 7fe271781700 1 --2- 172.21.15.6:0/14996378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe26c102730 0x7fe26c108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:07.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.250+0000 7fe271781700 1 --2- 172.21.15.6:0/14996378 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe26c100b90 0x7fe26c100fb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:07.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.250+0000 7fe271781700 1 --2- 172.21.15.6:0/14996378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe26c101d90 0x7fe26c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:07.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.250+0000 7fe271781700 1 -- 172.21.15.6:0/14996378 >> 172.21.15.6:0/14996378 conn(0x7fe26c0fc130 msgr2=0x7fe26c0fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:07.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.250+0000 7fe271781700 1 -- 172.21.15.6:0/14996378 shutdown_connections 2024-03-24T12:09:07.250 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.250+0000 7fe271781700 1 -- 172.21.15.6:0/14996378 wait complete. 2024-03-24T12:09:07.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.250+0000 7fe271781700 1 Processor -- start 2024-03-24T12:09:07.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.251+0000 7fe271781700 1 -- start start 2024-03-24T12:09:07.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.251+0000 7fe271781700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe26c100b90 0x7fe26c104b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:07.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.251+0000 7fe271781700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe26c101d90 0x7fe26c103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:07.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.251+0000 7fe271781700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe26c102730 0x7fe26c1036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:07.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.251+0000 7fe271781700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c103e30 con 0x7fe26c100b90 2024-03-24T12:09:07.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.251+0000 7fe271781700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c103fa0 con 0x7fe26c102730 2024-03-24T12:09:07.252 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.251+0000 7fe271781700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c1a13a0 con 0x7fe26c101d90 2024-03-24T12:09:07.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.251+0000 7fe26a7fc700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe26c101d90 0x7fe26c103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:07.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.251+0000 7fe26a7fc700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe26c101d90 0x7fe26c103170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:45670/0 (socket says 172.21.15.6:45670) 2024-03-24T12:09:07.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.251+0000 7fe26a7fc700 1 -- 172.21.15.6:0/3214389133 learned_addr learned my addr 172.21.15.6:0/3214389133 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:09:07.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.251+0000 7fe26affd700 1 --2- 172.21.15.6:0/3214389133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe26c100b90 0x7fe26c104b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:07.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.251+0000 7fe26b7fe700 1 --2- 172.21.15.6:0/3214389133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe26c102730 0x7fe26c1036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:07.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.251+0000 7fe26a7fc700 1 -- 172.21.15.6:0/3214389133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe26c102730 msgr2=0x7fe26c1036b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:07.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.251+0000 7fe26a7fc700 1 --2- 172.21.15.6:0/3214389133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe26c102730 0x7fe26c1036b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:07.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.251+0000 7fe26a7fc700 1 -- 172.21.15.6:0/3214389133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe26c100b90 msgr2=0x7fe26c104b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:07.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.251+0000 7fe26a7fc700 1 --2- 172.21.15.6:0/3214389133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe26c100b90 0x7fe26c104b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:07.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.251+0000 7fe26a7fc700 1 -- 172.21.15.6:0/3214389133 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe254009520 con 0x7fe26c101d90 2024-03-24T12:09:07.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.251+0000 7fe26a7fc700 1 --2- 172.21.15.6:0/3214389133 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe26c101d90 0x7fe26c103170 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe25400eb30 tx=0x7fe25400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:07.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.252+0000 7fe263fff700 1 -- 172.21.15.6:0/3214389133 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe254009880 con 0x7fe26c101d90 2024-03-24T12:09:07.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.252+0000 7fe271781700 1 -- 172.21.15.6:0/3214389133 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe25c009d00 con 0x7fe26c101d90 2024-03-24T12:09:07.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.252+0000 7fe263fff700 1 -- 172.21.15.6:0/3214389133 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe2540044a0 con 0x7fe26c101d90 2024-03-24T12:09:07.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.252+0000 7fe263fff700 1 -- 172.21.15.6:0/3214389133 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe254003ca0 con 0x7fe26c101d90 2024-03-24T12:09:07.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.252+0000 7fe271781700 1 -- 172.21.15.6:0/3214389133 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe26c1a1a40 con 0x7fe26c101d90 2024-03-24T12:09:07.254 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.253+0000 7fe271781700 1 -- 172.21.15.6:0/3214389133 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe26c068930 con 0x7fe26c101d90 2024-03-24T12:09:07.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.253+0000 7fe263fff700 1 -- 172.21.15.6:0/3214389133 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 15) v1 ==== 91079+0+0 (secure 0 0 0) 0x7fe254003e00 con 0x7fe26c101d90 2024-03-24T12:09:07.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.254+0000 7fe263fff700 1 --2- 172.21.15.6:0/3214389133 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fe25806d570 0x7fe25806fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:07.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.254+0000 7fe263fff700 1 -- 172.21.15.6:0/3214389133 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5320+0+0 (secure 0 0 0) 0x7fe254014070 con 0x7fe26c101d90 2024-03-24T12:09:07.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.254+0000 7fe26affd700 1 --2- 172.21.15.6:0/3214389133 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fe25806d570 0x7fe25806fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:07.255 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.254+0000 7fe26affd700 1 --2- 172.21.15.6:0/3214389133 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fe25806d570 0x7fe25806fa30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009fd0 tx=0x7fe25c0060c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:07.257 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.258+0000 7fe263fff700 1 -- 172.21.15.6:0/3214389133 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe254056bf0 con 0x7fe26c101d90 2024-03-24T12:09:07.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:06 smithi090 ceph-mon[33646]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-03-24T12:09:07.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:06 smithi090 ceph-mon[33646]: osdmap e53: 8 total, 7 up, 8 in 2024-03-24T12:09:07.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:06 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T12:09:07.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:06 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T12:09:07.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:06 smithi090 ceph-mon[33646]: osd.7 [v2:172.21.15.90:6824/1629523727,v1:172.21.15.90:6825/1629523727] boot 2024-03-24T12:09:07.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:06 smithi090 ceph-mon[33646]: osdmap e54: 8 total, 8 up, 8 in 2024-03-24T12:09:07.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:06 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T12:09:07.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.463+0000 7fe271781700 1 -- 172.21.15.6:0/3214389133 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fe26c1a1cc0 con 0x7fe26c101d90 2024-03-24T12:09:07.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.464+0000 7fe263fff700 1 -- 172.21.15.6:0/3214389133 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v55) v1 ==== 74+0+130 (secure 0 0 0) 0x7fe25405a2b0 con 0x7fe26c101d90 2024-03-24T12:09:07.463 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:09:07.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.466+0000 7fe271781700 1 -- 172.21.15.6:0/3214389133 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fe25806d570 msgr2=0x7fe25806fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:07.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.466+0000 7fe271781700 1 --2- 172.21.15.6:0/3214389133 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fe25806d570 0x7fe25806fa30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009fd0 tx=0x7fe25c0060c0 comp rx=0 tx=0).stop 2024-03-24T12:09:07.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.466+0000 7fe271781700 1 -- 172.21.15.6:0/3214389133 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe26c101d90 msgr2=0x7fe26c103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:07.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.466+0000 7fe271781700 1 --2- 172.21.15.6:0/3214389133 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe26c101d90 0x7fe26c103170 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe25400eb30 tx=0x7fe25400ee40 comp rx=0 tx=0).stop 2024-03-24T12:09:07.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.466+0000 7fe271781700 1 -- 172.21.15.6:0/3214389133 shutdown_connections 2024-03-24T12:09:07.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.466+0000 7fe271781700 1 --2- 172.21.15.6:0/3214389133 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fe25806d570 0x7fe25806fa30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:07.465 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.466+0000 7fe271781700 1 --2- 172.21.15.6:0/3214389133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe26c102730 0x7fe26c1036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:07.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.466+0000 7fe271781700 1 --2- 172.21.15.6:0/3214389133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe26c100b90 0x7fe26c104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:07.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.466+0000 7fe271781700 1 --2- 172.21.15.6:0/3214389133 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe26c101d90 0x7fe26c103170 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:07.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.466+0000 7fe271781700 1 -- 172.21.15.6:0/3214389133 >> 172.21.15.6:0/3214389133 conn(0x7fe26c0fc130 msgr2=0x7fe26c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:07.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.466+0000 7fe271781700 1 -- 172.21.15.6:0/3214389133 shutdown_connections 2024-03-24T12:09:07.466 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:07.466+0000 7fe271781700 1 -- 172.21.15.6:0/3214389133 wait complete. 2024-03-24T12:09:08.032 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:07 smithi006 ceph-mon[30108]: purged_snaps scrub starts 2024-03-24T12:09:08.033 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:07 smithi006 ceph-mon[30108]: purged_snaps scrub ok 2024-03-24T12:09:08.033 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:07 smithi006 ceph-mon[30108]: osdmap e55: 8 total, 8 up, 8 in 2024-03-24T12:09:08.033 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:07 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3214389133' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-24T12:09:08.033 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:07 smithi006 ceph-mon[30108]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:09:08.033 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:07 smithi006 ceph-mon[41734]: purged_snaps scrub starts 2024-03-24T12:09:08.033 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:07 smithi006 ceph-mon[41734]: purged_snaps scrub ok 2024-03-24T12:09:08.033 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:07 smithi006 ceph-mon[41734]: osdmap e55: 8 total, 8 up, 8 in 2024-03-24T12:09:08.033 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:07 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3214389133' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-24T12:09:08.033 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:07 smithi006 ceph-mon[41734]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:09:08.034 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":55,"num_osds":8,"num_up_osds":8,"osd_up_since":1711282146,"num_in_osds":8,"osd_in_since":1711282128,"num_remapped_pgs":1} 2024-03-24T12:09:08.035 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-24T12:09:08.111 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:07 smithi090 ceph-mon[33646]: purged_snaps scrub starts 2024-03-24T12:09:08.112 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:07 smithi090 ceph-mon[33646]: purged_snaps scrub ok 2024-03-24T12:09:08.112 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:07 smithi090 ceph-mon[33646]: osdmap e55: 8 total, 8 up, 8 in 2024-03-24T12:09:08.112 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:07 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3214389133' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-24T12:09:08.112 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:07 smithi090 ceph-mon[33646]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:09:08.276 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:09:09.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[30108]: osdmap e56: 8 total, 8 up, 8 in 2024-03-24T12:09:09.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[30108]: Detected new or changed devices on smithi090 2024-03-24T12:09:09.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:09.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:09.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:09:09.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:09:09.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:09:09.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:09:09.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[30108]: Adjusting osd_memory_target on smithi090 to 4272M 2024-03-24T12:09:09.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:09.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:09:09.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:09:09.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:09.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:09:09.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:09:09.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:09:09.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:09.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[41734]: osdmap e56: 8 total, 8 up, 8 in 2024-03-24T12:09:09.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[41734]: Detected new or changed devices on smithi090 2024-03-24T12:09:09.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:09.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:09.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:09:09.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:09:09.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:09:09.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:09:09.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[41734]: Adjusting osd_memory_target on smithi090 to 4272M 2024-03-24T12:09:09.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:09.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:09:09.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:09:09.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:09.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:09:09.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:09:09.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:09:09.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:09 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:09.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:09 smithi090 ceph-mon[33646]: osdmap e56: 8 total, 8 up, 8 in 2024-03-24T12:09:09.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:09 smithi090 ceph-mon[33646]: Detected new or changed devices on smithi090 2024-03-24T12:09:09.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:09 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:09.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:09 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:09.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:09 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:09:09.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:09 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:09:09.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:09 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:09:09.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:09 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:09:09.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:09 smithi090 ceph-mon[33646]: Adjusting osd_memory_target on smithi090 to 4272M 2024-03-24T12:09:09.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:09 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:09.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:09 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:09:09.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:09 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:09:09.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:09 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:09.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:09 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:09:09.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:09 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:09:09.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:09 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:09:09.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:09 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:09.999 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.998+0000 7f1f7f7a2700 1 -- 172.21.15.6:0/4046482955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f78105e90 msgr2=0x7f1f7810da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:10.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.998+0000 7f1f7f7a2700 1 --2- 172.21.15.6:0/4046482955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f78105e90 0x7f1f7810da60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009f90 tx=0x7f1f7000b330 comp rx=0 tx=0).stop 2024-03-24T12:09:10.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.998+0000 7f1f7f7a2700 1 -- 172.21.15.6:0/4046482955 shutdown_connections 2024-03-24T12:09:10.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.998+0000 7f1f7f7a2700 1 --2- 172.21.15.6:0/4046482955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f78105e90 0x7f1f7810da60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:10.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.998+0000 7f1f7f7a2700 1 --2- 172.21.15.6:0/4046482955 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f78100e00 0x7f1f781011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:10.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.998+0000 7f1f7f7a2700 1 --2- 172.21.15.6:0/4046482955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f781017b0 0x7f1f781057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:10.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.998+0000 7f1f7f7a2700 1 -- 172.21.15.6:0/4046482955 >> 172.21.15.6:0/4046482955 conn(0x7f1f78076b90 msgr2=0x7f1f78076fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:10.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.998+0000 7f1f7f7a2700 1 -- 172.21.15.6:0/4046482955 shutdown_connections 2024-03-24T12:09:10.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.998+0000 7f1f7f7a2700 1 -- 172.21.15.6:0/4046482955 wait complete. 2024-03-24T12:09:10.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.998+0000 7f1f7f7a2700 1 Processor -- start 2024-03-24T12:09:10.000 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.998+0000 7f1f7f7a2700 1 -- start start 2024-03-24T12:09:10.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.999+0000 7f1f7f7a2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f78100e00 0x7f1f78102850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:10.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.999+0000 7f1f7f7a2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f781017b0 0x7f1f78102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:10.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.999+0000 7f1f7f7a2700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f78105e90 0x7f1f781032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:10.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.999+0000 7f1f7f7a2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f781a5c80 con 0x7f1f781017b0 2024-03-24T12:09:10.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.999+0000 7f1f7f7a2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f781a5df0 con 0x7f1f78100e00 2024-03-24T12:09:10.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.999+0000 7f1f7f7a2700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f781a5fb0 con 0x7f1f78105e90 2024-03-24T12:09:10.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.999+0000 7f1f7dd3f700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f78105e90 0x7f1f781032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:10.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.999+0000 7f1f7cd3d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f781017b0 0x7f1f78102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:10.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.999+0000 7f1f7dd3f700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f78105e90 0x7f1f781032d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:45680/0 (socket says 172.21.15.6:45680) 2024-03-24T12:09:10.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.999+0000 7f1f7dd3f700 1 -- 172.21.15.6:0/3560155514 learned_addr learned my addr 172.21.15.6:0/3560155514 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:09:10.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.999+0000 7f1f7d53e700 1 --2- 172.21.15.6:0/3560155514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f78100e00 0x7f1f78102850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:10.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.999+0000 7f1f7dd3f700 1 -- 172.21.15.6:0/3560155514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f78100e00 msgr2=0x7f1f78102850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:10.001 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.999+0000 7f1f7dd3f700 1 --2- 172.21.15.6:0/3560155514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f78100e00 0x7f1f78102850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:10.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.999+0000 7f1f7dd3f700 1 -- 172.21.15.6:0/3560155514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f781017b0 msgr2=0x7f1f78102d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:10.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.999+0000 7f1f7dd3f700 1 --2- 172.21.15.6:0/3560155514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f781017b0 0x7f1f78102d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:10.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.999+0000 7f1f7dd3f700 1 -- 172.21.15.6:0/3560155514 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f740095f0 con 0x7f1f78105e90 2024-03-24T12:09:10.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.999+0000 7f1f7cd3d700 1 --2- 172.21.15.6:0/3560155514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f781017b0 0x7f1f78102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:09:10.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.999+0000 7f1f7dd3f700 1 --2- 172.21.15.6:0/3560155514 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f78105e90 0x7f1f781032d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1f70007c00 tx=0x7f1f70007c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:10.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.999+0000 7f1f6a7fc700 1 -- 172.21.15.6:0/3560155514 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7000e040 con 0x7f1f78105e90 2024-03-24T12:09:10.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.999+0000 7f1f6a7fc700 1 -- 172.21.15.6:0/3560155514 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1f70007e50 con 0x7f1f78105e90 2024-03-24T12:09:10.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:09.999+0000 7f1f7f7a2700 1 -- 172.21.15.6:0/3560155514 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f70009c40 con 0x7f1f78105e90 2024-03-24T12:09:10.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:10.000+0000 7f1f6a7fc700 1 -- 172.21.15.6:0/3560155514 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70005040 con 0x7f1f78105e90 2024-03-24T12:09:10.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:10.000+0000 7f1f7f7a2700 1 -- 172.21.15.6:0/3560155514 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f781a6510 con 0x7f1f78105e90 2024-03-24T12:09:10.002 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:10.001+0000 7f1f7f7a2700 1 -- 172.21.15.6:0/3560155514 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f7810a150 con 0x7f1f78105e90 2024-03-24T12:09:10.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:10.001+0000 7f1f6a7fc700 1 -- 172.21.15.6:0/3560155514 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 15) v1 ==== 91079+0+0 (secure 0 0 0) 0x7f1f7001b450 con 0x7f1f78105e90 2024-03-24T12:09:10.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:10.002+0000 7f1f6a7fc700 1 --2- 172.21.15.6:0/3560155514 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f1f6406d570 0x7f1f6406fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:10.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:10.002+0000 7f1f7d53e700 1 --2- 172.21.15.6:0/3560155514 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f1f6406d570 0x7f1f6406fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:10.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:10.002+0000 7f1f6a7fc700 1 -- 172.21.15.6:0/3560155514 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5287+0+0 (secure 0 0 0) 0x7f1f7008dea0 con 0x7f1f78105e90 2024-03-24T12:09:10.003 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:10.002+0000 7f1f7d53e700 1 --2- 172.21.15.6:0/3560155514 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f1f6406d570 0x7f1f6406fa30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1f74005fd0 tx=0x7f1f74009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:10.005 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:10.006+0000 7f1f6a7fc700 1 -- 172.21.15.6:0/3560155514 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1f70057e20 con 0x7f1f78105e90 2024-03-24T12:09:10.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:10.209+0000 7f1f7f7a2700 1 -- 172.21.15.6:0/3560155514 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f78103d80 con 0x7f1f78105e90 2024-03-24T12:09:10.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:10.210+0000 7f1f6a7fc700 1 -- 172.21.15.6:0/3560155514 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+13448 (secure 0 0 0) 0x7f1f7001bed0 con 0x7f1f78105e90 2024-03-24T12:09:10.210 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:09:10.210 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":57,"fsid":"66d0ad74-e9d6-11ee-95cd-87774f69a715","created":"2024-03-24T12:03:42.337185+0000","modified":"2024-03-24T12:09:09.460063+0000","last_up_change":"2024-03-24T12:09:06.451078+0000","last_in_change":"2024-03-24T12:08:48.739062+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T12:07:02.169336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3542a64b-e3c1-4a04-ba1c-bdfbcd4a0aa4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":195441355},{"type":"v1","addr":"172.21.15.6:6803","nonce":195441355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":195441355},{"type":"v1","addr":"172.21.15.6:6805","nonce":195441355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":195441355},{"type":"v1","addr":"172.21.15.6:6809","nonce":195441355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":195441355},{"type":"v1","addr":"172.21.15.6:6807","nonce":195441355}]},"public_addr":"172.21.15.6:6803/195441355","cluster_addr":"172.21.15.6:6805/195441355","heartbeat_back_addr":"172.21.15.6:6809/195441355","heartbeat_front_addr":"172.21.15.6:6807/195441355","state":["exists","up"]},{"osd":1,"uuid":"b00997af-0f11-4f04-ae58-81af568256cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3140266601},{"type":"v1","addr":"172.21.15.6:6811","nonce":3140266601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3140266601},{"type":"v1","addr":"172.21.15.6:6813","nonce":3140266601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3140266601},{"type":"v1","addr":"172.21.15.6:6817","nonce":3140266601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3140266601},{"type":"v1","addr":"172.21.15.6:6815","nonce":3140266601}]},"public_addr":"172.21.15.6:6811/3140266601","cluster_addr":"172.21.15.6:6813/3140266601","heartbeat_back_addr":"172.21.15.6:6817/3140266601","heartbeat_front_addr":"172.21.15.6:6815/3140266601","state":["exists","up"]},{"osd":2,"uuid":"eec34136-82f9-41eb-bbac-f0d6edf7e73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":4008388154},{"type":"v1","addr":"172.21.15.6:6819","nonce":4008388154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":4008388154},{"type":"v1","addr":"172.21.15.6:6821","nonce":4008388154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":4008388154},{"type":"v1","addr":"172.21.15.6:6825","nonce":4008388154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":4008388154},{"type":"v1","addr":"172.21.15.6:6823","nonce":4008388154}]},"public_addr":"172.21.15.6:6819/4008388154","cluster_addr":"172.21.15.6:6821/4008388154","heartbeat_back_addr":"172.21.15.6:6825/4008388154","heartbeat_front_addr":"172.21.15.6:6823/4008388154","state":["exists","up"]},{"osd":3,"uuid":"bd1c26cc-c121-4570-87ff-f432ecc53e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1341863884},{"type":"v1","addr":"172.21.15.6:6827","nonce":1341863884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1341863884},{"type":"v1","addr":"172.21.15.6:6829","nonce":1341863884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1341863884},{"type":"v1","addr":"172.21.15.6:6833","nonce":1341863884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1341863884},{"type":"v1","addr":"172.21.15.6:6831","nonce":1341863884}]},"public_addr":"172.21.15.6:6827/1341863884","cluster_addr":"172.21.15.6:6829/1341863884","heartbeat_back_addr":"172.21.15.6:6833/1341863884","heartbeat_front_addr":"172.21.15.6:6831/1341863884","state":["exists","up"]},{"osd":4,"uuid":"65a98356-0941-41e8-a10b-9d0f76212fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3147977284},{"type":"v1","addr":"172.21.15.90:6801","nonce":3147977284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3147977284},{"type":"v1","addr":"172.21.15.90:6803","nonce":3147977284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3147977284},{"type":"v1","addr":"172.21.15.90:6807","nonce":3147977284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3147977284},{"type":"v1","addr":"172.21.15.90:6805","nonce":3147977284}]},"public_addr":"172.21.15.90:6801/3147977284","cluster_addr":"172.21.15.90:6803/3147977284","heartbeat_back_addr":"172.21.15.90:6807/3147977284","heartbeat_front_addr":"172.21.15.90:6805/3147977284","state":["exists","up"]},{"osd":5,"uuid":"b143b33c-9447-4b8c-9991-5037f50789ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":3585923902},{"type":"v1","addr":"172.21.15.90:6809","nonce":3585923902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3585923902},{"type":"v1","addr":"172.21.15.90:6811","nonce":3585923902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3585923902},{"type":"v1","addr":"172.21.15.90:6815","nonce":3585923902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3585923902},{"type":"v1","addr":"172.21.15.90:6813","nonce":3585923902}]},"public_addr":"172.21.15.90:6809/3585923902","cluster_addr":"172.21.15.90:6811/3585923902","heartbeat_back_addr":"172.21.15.90:6815/3585923902","heartbeat_front_addr":"172.21.15.90:6813/3585923902","state":["exists","up"]},{"osd":6,"uuid":"43226b7c-1700-47e4-8d1b-7f765e8defda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":713247979},{"type":"v1","addr":"172.21.15.90:6817","nonce":713247979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":713247979},{"type":"v1","addr":"172.21.15.90:6819","nonce":713247979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":713247979},{"type":"v1","addr":"172.21.15.90:6823","nonce":713247979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":713247979},{"type":"v1","addr":"172.21.15.90:6821","nonce":713247979}]},"public_addr":"172.21.15.90:6817/713247979","cluster_addr":"172.21.15.90:6819/713247979","heartbeat_back_addr":"172.21.15.90:6823/713247979","heartbeat_front_addr":"172.21.15.90:6821/713247979","state":["exists","up"]},{"osd":7,"uuid":"fa0def8c-eda9-441c-8ee9-e0021c064bf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1629523727},{"type":"v1","addr":"172.21.15.90:6825","nonce":1629523727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1629523727},{"type":"v1","addr":"172.21.15.90:6827","nonce":1629523727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1629523727},{"type":"v1","addr":"172.21.15.90:6831","nonce":1629523727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1629523727},{"type":"v1","addr":"172.21.15.90:6829","nonce":1629523727}]},"public_addr":"172.21.15.90:6825/1629523727","cluster_addr":"172.21.15.90:6827/1629523727","heartbeat_back_addr":"172.21.15.90:6831/1629523727","heartbeat_front_addr":"172.21.15.90:6829/1629523727","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:06:08.730123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:06:33.621931+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:06:58.862101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:07:25.210912+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:07:49.154431+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:08:14.240635+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:08:39.558437+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:09:04.997129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1159789551":"2024-03-25T12:04:45.836851+0000","172.21.15.6:0/2683803593":"2024-03-25T12:04:45.836851+0000","172.21.15.6:6801/3010671792":"2024-03-25T12:04:45.836851+0000","172.21.15.6:6800/3010671792":"2024-03-25T12:04:45.836851+0000","172.21.15.6:6801/2054386689":"2024-03-25T12:04:18.792992+0000","172.21.15.6:6800/2054386689":"2024-03-25T12:04:18.792992+0000","172.21.15.6:0/897967172":"2024-03-25T12:04:18.792992+0000","172.21.15.6:0/2335021214":"2024-03-25T12:04:18.792992+0000","172.21.15.6:0/3268002752":"2024-03-25T12:04:18.792992+0000","172.21.15.6:0/1607559555":"2024-03-25T12:04:45.836851+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T12:09:10.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:10.212+0000 7f1f7f7a2700 1 -- 172.21.15.6:0/3560155514 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f1f6406d570 msgr2=0x7f1f6406fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:10.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:10.212+0000 7f1f7f7a2700 1 --2- 172.21.15.6:0/3560155514 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f1f6406d570 0x7f1f6406fa30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1f74005fd0 tx=0x7f1f74009310 comp rx=0 tx=0).stop 2024-03-24T12:09:10.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:10.212+0000 7f1f7f7a2700 1 -- 172.21.15.6:0/3560155514 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f78105e90 msgr2=0x7f1f781032d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:10.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:10.212+0000 7f1f7f7a2700 1 --2- 172.21.15.6:0/3560155514 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f78105e90 0x7f1f781032d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1f70007c00 tx=0x7f1f70007c30 comp rx=0 tx=0).stop 2024-03-24T12:09:10.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:10.213+0000 7f1f7f7a2700 1 -- 172.21.15.6:0/3560155514 shutdown_connections 2024-03-24T12:09:10.212 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:10.213+0000 7f1f7f7a2700 1 --2- 172.21.15.6:0/3560155514 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f1f6406d570 0x7f1f6406fa30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:10.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:10.213+0000 7f1f7f7a2700 1 --2- 172.21.15.6:0/3560155514 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1f78105e90 0x7f1f781032d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:10.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:10.213+0000 7f1f7f7a2700 1 --2- 172.21.15.6:0/3560155514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f78100e00 0x7f1f78102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:10.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:10.213+0000 7f1f7f7a2700 1 --2- 172.21.15.6:0/3560155514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f781017b0 0x7f1f78102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:10.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:10.213+0000 7f1f7f7a2700 1 -- 172.21.15.6:0/3560155514 >> 172.21.15.6:0/3560155514 conn(0x7f1f78076b90 msgr2=0x7f1f7806a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:10.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:10.213+0000 7f1f7f7a2700 1 -- 172.21.15.6:0/3560155514 shutdown_connections 2024-03-24T12:09:10.213 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:10.213+0000 7f1f7f7a2700 1 -- 172.21.15.6:0/3560155514 wait complete. 2024-03-24T12:09:10.814 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:10 smithi006 ceph-mon[30108]: osdmap e57: 8 total, 8 up, 8 in 2024-03-24T12:09:10.814 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:10 smithi006 ceph-mon[30108]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:09:10.814 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3560155514' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T12:09:10.815 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:10 smithi006 ceph-mon[41734]: osdmap e57: 8 total, 8 up, 8 in 2024-03-24T12:09:10.815 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:10 smithi006 ceph-mon[41734]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:09:10.815 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3560155514' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T12:09:10.843 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-24T12:07:02.169336+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-24T12:09:10.843 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-24T12:09:10.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:10 smithi090 ceph-mon[33646]: osdmap e57: 8 total, 8 up, 8 in 2024-03-24T12:09:10.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:10 smithi090 ceph-mon[33646]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:09:10.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3560155514' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T12:09:11.086 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:09:11.837 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:11 smithi006 ceph-mon[30108]: osdmap e58: 8 total, 8 up, 8 in 2024-03-24T12:09:11.837 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:11 smithi006 ceph-mon[41734]: osdmap e58: 8 total, 8 up, 8 in 2024-03-24T12:09:11.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:11 smithi090 ceph-mon[33646]: osdmap e58: 8 total, 8 up, 8 in 2024-03-24T12:09:12.531 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:12 smithi006 ceph-mon[30108]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:09:12.531 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:12 smithi006 ceph-mon[41734]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:09:12.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.760+0000 7f3a4da5f700 1 -- 172.21.15.6:0/3846801616 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a48107d00 msgr2=0x7f3a481080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:12.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.760+0000 7f3a4da5f700 1 --2- 172.21.15.6:0/3846801616 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a48107d00 0x7f3a481080e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3a40009990 tx=0x7f3a40009ca0 comp rx=0 tx=0).stop 2024-03-24T12:09:12.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.760+0000 7f3a4da5f700 1 -- 172.21.15.6:0/3846801616 shutdown_connections 2024-03-24T12:09:12.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.760+0000 7f3a4da5f700 1 --2- 172.21.15.6:0/3846801616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a481026b0 0x7f3a4810eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:12.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.760+0000 7f3a4da5f700 1 --2- 172.21.15.6:0/3846801616 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a48107d00 0x7f3a481080e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:12.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.760+0000 7f3a4da5f700 1 --2- 172.21.15.6:0/3846801616 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a48101cf0 0x7f3a48102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:12.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.760+0000 7f3a4da5f700 1 -- 172.21.15.6:0/3846801616 >> 172.21.15.6:0/3846801616 conn(0x7f3a48078dd0 msgr2=0x7f3a480791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:12.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.760+0000 7f3a4da5f700 1 -- 172.21.15.6:0/3846801616 shutdown_connections 2024-03-24T12:09:12.760 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.760+0000 7f3a4da5f700 1 -- 172.21.15.6:0/3846801616 wait complete. 2024-03-24T12:09:12.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.761+0000 7f3a4da5f700 1 Processor -- start 2024-03-24T12:09:12.761 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.761+0000 7f3a4da5f700 1 -- start start 2024-03-24T12:09:12.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.761+0000 7f3a4da5f700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a48101cf0 0x7f3a48197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:12.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.761+0000 7f3a4da5f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a481026b0 0x7f3a48197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:12.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.761+0000 7f3a4da5f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a48107d00 0x7f3a4819bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:12.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.761+0000 7f3a4da5f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4819c520 con 0x7f3a48107d00 2024-03-24T12:09:12.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.761+0000 7f3a4da5f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4819c690 con 0x7f3a481026b0 2024-03-24T12:09:12.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.761+0000 7f3a4da5f700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4819c850 con 0x7f3a48101cf0 2024-03-24T12:09:12.763 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.761+0000 7f3a46ffd700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a48101cf0 0x7f3a48197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:12.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.761+0000 7f3a46ffd700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a48101cf0 0x7f3a48197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:45702/0 (socket says 172.21.15.6:45702) 2024-03-24T12:09:12.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.761+0000 7f3a46ffd700 1 -- 172.21.15.6:0/495395495 learned_addr learned my addr 172.21.15.6:0/495395495 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:09:12.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.761+0000 7f3a477fe700 1 --2- 172.21.15.6:0/495395495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a48107d00 0x7f3a4819bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:12.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.761+0000 7f3a467fc700 1 --2- 172.21.15.6:0/495395495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a481026b0 0x7f3a48197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:12.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.761+0000 7f3a46ffd700 1 -- 172.21.15.6:0/495395495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a481026b0 msgr2=0x7f3a48197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:12.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.762+0000 7f3a46ffd700 1 --2- 172.21.15.6:0/495395495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a481026b0 0x7f3a48197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:12.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.762+0000 7f3a46ffd700 1 -- 172.21.15.6:0/495395495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a48107d00 msgr2=0x7f3a4819bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:12.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.762+0000 7f3a46ffd700 1 --2- 172.21.15.6:0/495395495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a48107d00 0x7f3a4819bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:12.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.762+0000 7f3a46ffd700 1 -- 172.21.15.6:0/495395495 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a30009520 con 0x7f3a48101cf0 2024-03-24T12:09:12.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.762+0000 7f3a477fe700 1 --2- 172.21.15.6:0/495395495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a48107d00 0x7f3a4819bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:09:12.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.762+0000 7f3a46ffd700 1 --2- 172.21.15.6:0/495395495 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a48101cf0 0x7f3a48197640 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3a40005910 tx=0x7f3a40004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:12.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.762+0000 7f3a4ca5d700 1 -- 172.21.15.6:0/495395495 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4001d070 con 0x7f3a48101cf0 2024-03-24T12:09:12.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.762+0000 7f3a4ca5d700 1 -- 172.21.15.6:0/495395495 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f3a40022470 con 0x7f3a48101cf0 2024-03-24T12:09:12.764 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.762+0000 7f3a4da5f700 1 -- 172.21.15.6:0/495395495 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a400095f0 con 0x7f3a48101cf0 2024-03-24T12:09:12.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.762+0000 7f3a4ca5d700 1 -- 172.21.15.6:0/495395495 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a40017650 con 0x7f3a48101cf0 2024-03-24T12:09:12.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.762+0000 7f3a4da5f700 1 -- 172.21.15.6:0/495395495 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a481a41c0 con 0x7f3a48101cf0 2024-03-24T12:09:12.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.763+0000 7f3a4da5f700 1 -- 172.21.15.6:0/495395495 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a48068930 con 0x7f3a48101cf0 2024-03-24T12:09:12.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.763+0000 7f3a4ca5d700 1 -- 172.21.15.6:0/495395495 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 15) v1 ==== 91079+0+0 (secure 0 0 0) 0x7f3a4000fac0 con 0x7f3a48101cf0 2024-03-24T12:09:12.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.764+0000 7f3a4ca5d700 1 --2- 172.21.15.6:0/495395495 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f3a3406d570 0x7f3a3406fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:12.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.764+0000 7f3a4ca5d700 1 -- 172.21.15.6:0/495395495 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5287+0+0 (secure 0 0 0) 0x7f3a4008de30 con 0x7f3a48101cf0 2024-03-24T12:09:12.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.764+0000 7f3a467fc700 1 --2- 172.21.15.6:0/495395495 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f3a3406d570 0x7f3a3406fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:12.765 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.764+0000 7f3a467fc700 1 --2- 172.21.15.6:0/495395495 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f3a3406d570 0x7f3a3406fa30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3a4806b400 tx=0x7f3a30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:12.767 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.768+0000 7f3a4ca5d700 1 -- 172.21.15.6:0/495395495 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3a40057db0 con 0x7f3a48101cf0 2024-03-24T12:09:12.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:12 smithi090 ceph-mon[33646]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:09:12.968 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.969+0000 7f3a4da5f700 1 -- 172.21.15.6:0/495395495 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3a48111260 con 0x7f3a48101cf0 2024-03-24T12:09:12.969 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.969+0000 7f3a4ca5d700 1 -- 172.21.15.6:0/495395495 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3a4005b470 con 0x7f3a48101cf0 2024-03-24T12:09:12.969 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2024-03-24T12:09:12.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.971+0000 7f3a4da5f700 1 -- 172.21.15.6:0/495395495 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f3a3406d570 msgr2=0x7f3a3406fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:12.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.971+0000 7f3a4da5f700 1 --2- 172.21.15.6:0/495395495 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f3a3406d570 0x7f3a3406fa30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3a4806b400 tx=0x7f3a30009260 comp rx=0 tx=0).stop 2024-03-24T12:09:12.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.971+0000 7f3a4da5f700 1 -- 172.21.15.6:0/495395495 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a48101cf0 msgr2=0x7f3a48197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:12.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.971+0000 7f3a4da5f700 1 --2- 172.21.15.6:0/495395495 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a48101cf0 0x7f3a48197640 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3a40005910 tx=0x7f3a40004730 comp rx=0 tx=0).stop 2024-03-24T12:09:12.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.972+0000 7f3a4da5f700 1 -- 172.21.15.6:0/495395495 shutdown_connections 2024-03-24T12:09:12.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.972+0000 7f3a4da5f700 1 --2- 172.21.15.6:0/495395495 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f3a3406d570 0x7f3a3406fa30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:12.971 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.972+0000 7f3a4da5f700 1 --2- 172.21.15.6:0/495395495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a48107d00 0x7f3a4819bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:12.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.972+0000 7f3a4da5f700 1 --2- 172.21.15.6:0/495395495 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a48101cf0 0x7f3a48197640 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:12.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.972+0000 7f3a4da5f700 1 --2- 172.21.15.6:0/495395495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a481026b0 0x7f3a48197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:12.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.972+0000 7f3a4da5f700 1 -- 172.21.15.6:0/495395495 >> 172.21.15.6:0/495395495 conn(0x7f3a48078dd0 msgr2=0x7f3a481058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:12.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.972+0000 7f3a4da5f700 1 -- 172.21.15.6:0/495395495 shutdown_connections 2024-03-24T12:09:12.972 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:12.972+0000 7f3a4da5f700 1 -- 172.21.15.6:0/495395495 wait complete. 2024-03-24T12:09:13.646 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:13 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/495395495' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T12:09:13.646 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:13 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/495395495' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T12:09:13.677 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi006 2024-03-24T12:09:13.677 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi006=foo.a' 2024-03-24T12:09:13.753 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:13 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/495395495' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-24T12:09:13.933 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T12:09:14.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:14 smithi090 ceph-mon[33646]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-24T12:09:14.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:14 smithi006 ceph-mon[30108]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-24T12:09:14.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:14 smithi006 ceph-mon[41734]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-24T12:09:15.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.578+0000 7f8e52753700 1 -- 172.21.15.90:0/3050445853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e4c0ffd20 msgr2=0x7f8e4c1001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:15.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.578+0000 7f8e52753700 1 --2- 172.21.15.90:0/3050445853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e4c0ffd20 0x7f8e4c1001a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8e40009990 tx=0x7f8e40009ca0 comp rx=0 tx=0).stop 2024-03-24T12:09:15.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.578+0000 7f8e52753700 1 -- 172.21.15.90:0/3050445853 shutdown_connections 2024-03-24T12:09:15.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.578+0000 7f8e52753700 1 --2- 172.21.15.90:0/3050445853 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e4c1006e0 0x7f8e4c10a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:15.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.578+0000 7f8e52753700 1 --2- 172.21.15.90:0/3050445853 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8e4c1032a0 0x7f8e4c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:15.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.578+0000 7f8e52753700 1 --2- 172.21.15.90:0/3050445853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e4c0ffd20 0x7f8e4c1001a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:15.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.578+0000 7f8e52753700 1 -- 172.21.15.90:0/3050445853 >> 172.21.15.90:0/3050445853 conn(0x7f8e4c0fb8e0 msgr2=0x7f8e4c0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:15.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.578+0000 7f8e52753700 1 -- 172.21.15.90:0/3050445853 shutdown_connections 2024-03-24T12:09:15.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.578+0000 7f8e52753700 1 -- 172.21.15.90:0/3050445853 wait complete. 2024-03-24T12:09:15.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.578+0000 7f8e52753700 1 Processor -- start 2024-03-24T12:09:15.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.578+0000 7f8e52753700 1 -- start start 2024-03-24T12:09:15.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.579+0000 7f8e52753700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e4c0ffd20 0x7f8e4c1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:15.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.579+0000 7f8e52753700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e4c1006e0 0x7f8e4c1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:15.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.579+0000 7f8e52753700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8e4c1032a0 0x7f8e4c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:15.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.579+0000 7f8e52753700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c19ee30 con 0x7f8e4c1006e0 2024-03-24T12:09:15.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.579+0000 7f8e52753700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c19ef70 con 0x7f8e4c0ffd20 2024-03-24T12:09:15.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.579+0000 7f8e52753700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c19f100 con 0x7f8e4c1032a0 2024-03-24T12:09:15.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.579+0000 7f8e4bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e4c0ffd20 0x7f8e4c1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:15.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.579+0000 7f8e4bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e4c0ffd20 0x7f8e4c1972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51662/0 (socket says 172.21.15.90:51662) 2024-03-24T12:09:15.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.579+0000 7f8e4bfff700 1 -- 172.21.15.90:0/607543366 learned_addr learned my addr 172.21.15.90:0/607543366 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-24T12:09:15.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.579+0000 7f8e4b7fe700 1 --2- 172.21.15.90:0/607543366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e4c1006e0 0x7f8e4c1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:15.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.579+0000 7f8e50cf0700 1 --2- 172.21.15.90:0/607543366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8e4c1032a0 0x7f8e4c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:15.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.579+0000 7f8e4bfff700 1 -- 172.21.15.90:0/607543366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8e4c1032a0 msgr2=0x7f8e4c19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:15.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.579+0000 7f8e4bfff700 1 --2- 172.21.15.90:0/607543366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8e4c1032a0 0x7f8e4c19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:15.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.579+0000 7f8e4bfff700 1 -- 172.21.15.90:0/607543366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e4c1006e0 msgr2=0x7f8e4c1977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:15.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.579+0000 7f8e4bfff700 1 --2- 172.21.15.90:0/607543366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e4c1006e0 0x7f8e4c1977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:15.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.579+0000 7f8e4bfff700 1 -- 172.21.15.90:0/607543366 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e400095f0 con 0x7f8e4c0ffd20 2024-03-24T12:09:15.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.579+0000 7f8e4bfff700 1 --2- 172.21.15.90:0/607543366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e4c0ffd20 0x7f8e4c1972a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00b500 tx=0x7f8e3c00b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:15.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.579+0000 7f8e497fa700 1 -- 172.21.15.90:0/607543366 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c011830 con 0x7f8e4c0ffd20 2024-03-24T12:09:15.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.579+0000 7f8e52753700 1 -- 172.21.15.90:0/607543366 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e4c19f3e0 con 0x7f8e4c0ffd20 2024-03-24T12:09:15.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.579+0000 7f8e497fa700 1 -- 172.21.15.90:0/607543366 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f8e3c011e10 con 0x7f8e4c0ffd20 2024-03-24T12:09:15.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.579+0000 7f8e497fa700 1 -- 172.21.15.90:0/607543366 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c010c60 con 0x7f8e4c0ffd20 2024-03-24T12:09:15.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.579+0000 7f8e52753700 1 -- 172.21.15.90:0/607543366 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4c19f930 con 0x7f8e4c0ffd20 2024-03-24T12:09:15.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.580+0000 7f8e52753700 1 -- 172.21.15.90:0/607543366 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e4c108a20 con 0x7f8e4c0ffd20 2024-03-24T12:09:15.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.581+0000 7f8e497fa700 1 -- 172.21.15.90:0/607543366 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 15) v1 ==== 91079+0+0 (secure 0 0 0) 0x7f8e3c011990 con 0x7f8e4c0ffd20 2024-03-24T12:09:15.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.581+0000 7f8e497fa700 1 --2- 172.21.15.90:0/607543366 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f8e3406d570 0x7f8e3406fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:15.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.581+0000 7f8e497fa700 1 -- 172.21.15.90:0/607543366 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5287+0+0 (secure 0 0 0) 0x7f8e3c08c3c0 con 0x7f8e4c0ffd20 2024-03-24T12:09:15.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.581+0000 7f8e4b7fe700 1 --2- 172.21.15.90:0/607543366 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f8e3406d570 0x7f8e3406fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:15.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.582+0000 7f8e4b7fe700 1 --2- 172.21.15.90:0/607543366 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f8e3406d570 0x7f8e3406fa30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8e40006010 tx=0x7f8e40003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:15.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.586+0000 7f8e497fa700 1 -- 172.21.15.90:0/607543366 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8e3c0562c0 con 0x7f8e4c0ffd20 2024-03-24T12:09:15.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.788+0000 7f8e52753700 1 -- 172.21.15.90:0/607543366 --> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi006=foo.a", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e4c10cd40 con 0x7f8e3406d570 2024-03-24T12:09:15.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.795+0000 7f8e497fa700 1 -- 172.21.15.90:0/607543366 <== mgr.14152 v2:172.21.15.6:6800/3847865179 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (secure 0 0 0) 0x7f8e4c10cd40 con 0x7f8e3406d570 2024-03-24T12:09:15.795 INFO:teuthology.orchestra.run.smithi090.stdout:Scheduled rgw.foo.a update... 2024-03-24T12:09:15.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.797+0000 7f8e52753700 1 -- 172.21.15.90:0/607543366 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f8e3406d570 msgr2=0x7f8e3406fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:15.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.797+0000 7f8e52753700 1 --2- 172.21.15.90:0/607543366 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f8e3406d570 0x7f8e3406fa30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8e40006010 tx=0x7f8e40003680 comp rx=0 tx=0).stop 2024-03-24T12:09:15.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.797+0000 7f8e52753700 1 -- 172.21.15.90:0/607543366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e4c0ffd20 msgr2=0x7f8e4c1972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:15.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.797+0000 7f8e52753700 1 --2- 172.21.15.90:0/607543366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e4c0ffd20 0x7f8e4c1972a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00b500 tx=0x7f8e3c00b810 comp rx=0 tx=0).stop 2024-03-24T12:09:15.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.797+0000 7f8e52753700 1 -- 172.21.15.90:0/607543366 shutdown_connections 2024-03-24T12:09:15.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.797+0000 7f8e52753700 1 --2- 172.21.15.90:0/607543366 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f8e3406d570 0x7f8e3406fa30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:15.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.797+0000 7f8e52753700 1 --2- 172.21.15.90:0/607543366 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8e4c1032a0 0x7f8e4c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:15.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.797+0000 7f8e52753700 1 --2- 172.21.15.90:0/607543366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e4c0ffd20 0x7f8e4c1972a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:15.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.797+0000 7f8e52753700 1 --2- 172.21.15.90:0/607543366 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e4c1006e0 0x7f8e4c1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:15.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.798+0000 7f8e52753700 1 -- 172.21.15.90:0/607543366 >> 172.21.15.90:0/607543366 conn(0x7f8e4c0fb8e0 msgr2=0x7f8e4c0fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:15.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.798+0000 7f8e52753700 1 -- 172.21.15.90:0/607543366 shutdown_connections 2024-03-24T12:09:15.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:15.798+0000 7f8e52753700 1 -- 172.21.15.90:0/607543366 wait complete. 2024-03-24T12:09:16.443 DEBUG:teuthology.orchestra.run.smithi006:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@rgw.foo.a.service 2024-03-24T12:09:16.446 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi090 2024-03-24T12:09:16.446 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph osd pool create datapool 3 3 replicated 2024-03-24T12:09:16.682 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T12:09:17.053 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:16 smithi090 ceph-mon[33646]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi006=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:09:17.053 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:16 smithi090 ceph-mon[33646]: Saving service rgw.foo.a spec with placement smithi006=foo.a;count:1 2024-03-24T12:09:17.053 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:16 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:17.053 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:16 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:09:17.053 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:16 smithi090 ceph-mon[33646]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-24T12:09:17.053 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:16 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:09:17.054 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:16 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:09:17.054 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:16 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:17.054 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:16 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T12:09:17.054 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:16 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-24T12:09:17.054 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:16 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:17.054 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:16 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:09:17.057 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[30108]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi006=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:09:17.057 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[30108]: Saving service rgw.foo.a spec with placement smithi006=foo.a;count:1 2024-03-24T12:09:17.057 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:17.057 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:09:17.057 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[30108]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-24T12:09:17.058 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:09:17.058 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:09:17.058 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:17.058 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T12:09:17.058 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-24T12:09:17.058 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:17.058 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:09:17.058 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[41734]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi006=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:09:17.058 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[41734]: Saving service rgw.foo.a spec with placement smithi006=foo.a;count:1 2024-03-24T12:09:17.058 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:17.058 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:09:17.058 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[41734]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-24T12:09:17.059 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:09:17.059 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:09:17.059 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:17.059 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-24T12:09:17.059 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-24T12:09:17.059 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:17.059 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:16 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:09:17.994 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:17 smithi090 ceph-mon[33646]: Deploying daemon rgw.foo.a on smithi006 2024-03-24T12:09:18.045 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:17 smithi006 ceph-mon[30108]: Deploying daemon rgw.foo.a on smithi006 2024-03-24T12:09:18.045 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:17 smithi006 ceph-mon[41734]: Deploying daemon rgw.foo.a on smithi006 2024-03-24T12:09:18.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.218+0000 7f80cd4b1700 1 -- 172.21.15.90:0/4159990296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80c80fffb0 msgr2=0x7f80c81003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:18.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.218+0000 7f80cd4b1700 1 --2- 172.21.15.90:0/4159990296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80c80fffb0 0x7f80c81003d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009990 tx=0x7f80b8009ca0 comp rx=0 tx=0).stop 2024-03-24T12:09:18.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.218+0000 7f80cd4b1700 1 -- 172.21.15.90:0/4159990296 shutdown_connections 2024-03-24T12:09:18.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.218+0000 7f80cd4b1700 1 --2- 172.21.15.90:0/4159990296 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80c8103930 0x7f80c810a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:18.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.218+0000 7f80cd4b1700 1 --2- 172.21.15.90:0/4159990296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80c80fffb0 0x7f80c81003d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:18.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.218+0000 7f80cd4b1700 1 --2- 172.21.15.90:0/4159990296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80c8102f90 0x7f80c81033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:18.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.218+0000 7f80cd4b1700 1 -- 172.21.15.90:0/4159990296 >> 172.21.15.90:0/4159990296 conn(0x7f80c80fb910 msgr2=0x7f80c80fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:18.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.218+0000 7f80cd4b1700 1 -- 172.21.15.90:0/4159990296 shutdown_connections 2024-03-24T12:09:18.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.218+0000 7f80cd4b1700 1 -- 172.21.15.90:0/4159990296 wait complete. 2024-03-24T12:09:18.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.218+0000 7f80cd4b1700 1 Processor -- start 2024-03-24T12:09:18.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.219+0000 7f80cd4b1700 1 -- start start 2024-03-24T12:09:18.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.219+0000 7f80cd4b1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80c80fffb0 0x7f80c8197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:18.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.219+0000 7f80cd4b1700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80c8102f90 0x7f80c8197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:18.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.219+0000 7f80cd4b1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80c8103930 0x7f80c819e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:18.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.219+0000 7f80cd4b1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c819eda0 con 0x7f80c8103930 2024-03-24T12:09:18.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.219+0000 7f80cd4b1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c819eee0 con 0x7f80c80fffb0 2024-03-24T12:09:18.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.219+0000 7f80cd4b1700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c819f070 con 0x7f80c8102f90 2024-03-24T12:09:18.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.219+0000 7f80c6ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80c80fffb0 0x7f80c8197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:18.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.219+0000 7f80c6ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80c80fffb0 0x7f80c8197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51690/0 (socket says 172.21.15.90:51690) 2024-03-24T12:09:18.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.219+0000 7f80c6ffd700 1 -- 172.21.15.90:0/1916519784 learned_addr learned my addr 172.21.15.90:0/1916519784 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-24T12:09:18.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.219+0000 7f80be7fc700 1 --2- 172.21.15.90:0/1916519784 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80c8102f90 0x7f80c8197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:18.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.219+0000 7f80c77fe700 1 --2- 172.21.15.90:0/1916519784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80c8103930 0x7f80c819e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:18.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.219+0000 7f80c6ffd700 1 -- 172.21.15.90:0/1916519784 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80c8102f90 msgr2=0x7f80c8197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:18.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.219+0000 7f80c6ffd700 1 --2- 172.21.15.90:0/1916519784 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80c8102f90 0x7f80c8197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:18.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.219+0000 7f80c6ffd700 1 -- 172.21.15.90:0/1916519784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80c8103930 msgr2=0x7f80c819e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:18.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.219+0000 7f80c6ffd700 1 --2- 172.21.15.90:0/1916519784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80c8103930 0x7f80c819e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:18.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.219+0000 7f80c6ffd700 1 -- 172.21.15.90:0/1916519784 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b80095f0 con 0x7f80c80fffb0 2024-03-24T12:09:18.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.219+0000 7f80c6ffd700 1 --2- 172.21.15.90:0/1916519784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80c80fffb0 0x7f80c8197210 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f80b80036f0 tx=0x7f80b800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:18.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.219+0000 7f80c4ff9700 1 -- 172.21.15.90:0/1916519784 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b801d070 con 0x7f80c80fffb0 2024-03-24T12:09:18.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.219+0000 7f80cd4b1700 1 -- 172.21.15.90:0/1916519784 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c819f2f0 con 0x7f80c80fffb0 2024-03-24T12:09:18.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.219+0000 7f80c4ff9700 1 -- 172.21.15.90:0/1916519784 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f80b8022470 con 0x7f80c80fffb0 2024-03-24T12:09:18.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.219+0000 7f80cd4b1700 1 -- 172.21.15.90:0/1916519784 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c819f760 con 0x7f80c80fffb0 2024-03-24T12:09:18.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.220+0000 7f80c4ff9700 1 -- 172.21.15.90:0/1916519784 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8003a20 con 0x7f80c80fffb0 2024-03-24T12:09:18.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.220+0000 7f80cd4b1700 1 -- 172.21.15.90:0/1916519784 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c8108970 con 0x7f80c80fffb0 2024-03-24T12:09:18.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.221+0000 7f80c4ff9700 1 -- 172.21.15.90:0/1916519784 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 15) v1 ==== 91079+0+0 (secure 0 0 0) 0x7f80b8022900 con 0x7f80c80fffb0 2024-03-24T12:09:18.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.221+0000 7f80c4ff9700 1 --2- 172.21.15.90:0/1916519784 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f80b406d570 0x7f80b406fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:18.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.221+0000 7f80c4ff9700 1 -- 172.21.15.90:0/1916519784 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5287+0+0 (secure 0 0 0) 0x7f80b808e090 con 0x7f80c80fffb0 2024-03-24T12:09:18.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.221+0000 7f80be7fc700 1 --2- 172.21.15.90:0/1916519784 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f80b406d570 0x7f80b406fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:18.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.222+0000 7f80be7fc700 1 --2- 172.21.15.90:0/1916519784 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f80b406d570 0x7f80b406fa30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f80b0005950 tx=0x7f80b000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:18.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.226+0000 7f80c4ff9700 1 -- 172.21.15.90:0/1916519784 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f80b80580b0 con 0x7f80c80fffb0 2024-03-24T12:09:18.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.418+0000 7f80cd4b1700 1 -- 172.21.15.90:0/1916519784 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) v1 -- 0x7f80c810ca90 con 0x7f80c80fffb0 2024-03-24T12:09:18.711 INFO:journalctl@ceph.rgw.foo.a.smithi006.stdout:Mar 24 12:09:18 smithi006 systemd[1]: Starting Ceph rgw.foo.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:09:18.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.811+0000 7f80c4ff9700 1 -- 172.21.15.90:0/1916519784 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'datapool' created v59) v1 ==== 160+0+0 (secure 0 0 0) 0x7f80b805b770 con 0x7f80c80fffb0 2024-03-24T12:09:18.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.813+0000 7f80cd4b1700 1 -- 172.21.15.90:0/1916519784 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f80b406d570 msgr2=0x7f80b406fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:18.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.813+0000 7f80cd4b1700 1 --2- 172.21.15.90:0/1916519784 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f80b406d570 0x7f80b406fa30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f80b0005950 tx=0x7f80b000a300 comp rx=0 tx=0).stop 2024-03-24T12:09:18.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.813+0000 7f80cd4b1700 1 -- 172.21.15.90:0/1916519784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80c80fffb0 msgr2=0x7f80c8197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:18.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.813+0000 7f80cd4b1700 1 --2- 172.21.15.90:0/1916519784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80c80fffb0 0x7f80c8197210 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f80b80036f0 tx=0x7f80b800f690 comp rx=0 tx=0).stop 2024-03-24T12:09:18.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.813+0000 7f80cd4b1700 1 -- 172.21.15.90:0/1916519784 shutdown_connections 2024-03-24T12:09:18.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.813+0000 7f80cd4b1700 1 --2- 172.21.15.90:0/1916519784 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f80b406d570 0x7f80b406fa30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:18.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.813+0000 7f80cd4b1700 1 --2- 172.21.15.90:0/1916519784 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80c8103930 0x7f80c819e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:18.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.813+0000 7f80cd4b1700 1 --2- 172.21.15.90:0/1916519784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80c80fffb0 0x7f80c8197210 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:18.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.813+0000 7f80cd4b1700 1 --2- 172.21.15.90:0/1916519784 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80c8102f90 0x7f80c8197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:18.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.813+0000 7f80cd4b1700 1 -- 172.21.15.90:0/1916519784 >> 172.21.15.90:0/1916519784 conn(0x7f80c80fb910 msgr2=0x7f80c80fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:18.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.813+0000 7f80cd4b1700 1 -- 172.21.15.90:0/1916519784 shutdown_connections 2024-03-24T12:09:18.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:18.813+0000 7f80cd4b1700 1 -- 172.21.15.90:0/1916519784 wait complete. 2024-03-24T12:09:18.813 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'datapool' created 2024-03-24T12:09:19.051 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:18 smithi006 ceph-mon[30108]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-24T12:09:19.051 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.90:0/1916519784' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-24T12:09:19.051 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-24T12:09:19.051 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:18 smithi006 ceph-mon[41734]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-24T12:09:19.051 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.90:0/1916519784' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-24T12:09:19.051 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-24T12:09:19.052 INFO:journalctl@ceph.rgw.foo.a.smithi006.stdout:Mar 24 12:09:18 smithi006 podman[69920]: 2024-03-24 12:09:18.627456906 +0000 UTC m=+0.017591011 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-24T12:09:19.052 INFO:journalctl@ceph.rgw.foo.a.smithi006.stdout:Mar 24 12:09:18 smithi006 podman[69920]: 2024-03-24 12:09:18.732843663 +0000 UTC m=+0.122977798 container create f208de7bada3e5f1802ecd744051240e1b954900b543e6206ec627a45e1740ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-rgw-foo-a, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-24T12:09:19.052 INFO:journalctl@ceph.rgw.foo.a.smithi006.stdout:Mar 24 12:09:18 smithi006 podman[69920]: 2024-03-24 12:09:18.952242099 +0000 UTC m=+0.342376192 container init f208de7bada3e5f1802ecd744051240e1b954900b543e6206ec627a45e1740ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-rgw-foo-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-24T12:09:19.052 INFO:journalctl@ceph.rgw.foo.a.smithi006.stdout:Mar 24 12:09:18 smithi006 podman[69920]: 2024-03-24 12:09:18.95796023 +0000 UTC m=+0.348094323 container start f208de7bada3e5f1802ecd744051240e1b954900b543e6206ec627a45e1740ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-rgw-foo-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T12:09:19.107 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:18 smithi090 ceph-mon[33646]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-24T12:09:19.107 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.90:0/1916519784' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-24T12:09:19.107 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-24T12:09:19.443 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- rbd pool init datapool 2024-03-24T12:09:19.460 INFO:journalctl@ceph.rgw.foo.a.smithi006.stdout:Mar 24 12:09:19 smithi006 bash[69920]: f208de7bada3e5f1802ecd744051240e1b954900b543e6206ec627a45e1740ec 2024-03-24T12:09:19.460 INFO:journalctl@ceph.rgw.foo.a.smithi006.stdout:Mar 24 12:09:19 smithi006 systemd[1]: Started Ceph rgw.foo.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:09:19.677 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T12:09:20.111 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-03-24T12:09:20.112 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:19 smithi090 ceph-mon[33646]: osdmap e59: 8 total, 8 up, 8 in 2024-03-24T12:09:20.112 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:19 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:20.112 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:19 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:20.112 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:19 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:20.112 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:19 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:20.112 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:19 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:20.112 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:19 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:09:20.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-03-24T12:09:20.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:19 smithi006 ceph-mon[30108]: osdmap e59: 8 total, 8 up, 8 in 2024-03-24T12:09:20.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:19 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:20.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:19 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:20.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:19 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:20.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:19 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:20.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:19 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:20.170 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:19 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:09:20.170 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-03-24T12:09:20.171 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:19 smithi006 ceph-mon[41734]: osdmap e59: 8 total, 8 up, 8 in 2024-03-24T12:09:20.171 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:19 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:20.171 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:19 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:20.171 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:19 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:20.171 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:19 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:20.171 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:19 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:20.171 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:19 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:09:20.850 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:20 smithi090 ceph-mon[33646]: Saving service rgw.foo.a spec with placement smithi006=foo.a;count:1 2024-03-24T12:09:20.851 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:20 smithi090 ceph-mon[33646]: osdmap e60: 8 total, 8 up, 8 in 2024-03-24T12:09:20.851 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:20 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/380256656' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-24T12:09:20.851 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:20 smithi090 ceph-mon[33646]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-24T12:09:20.851 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:20 smithi090 ceph-mon[33646]: pgmap v187: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-24T12:09:20.851 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:20 smithi090 ceph-mon[33646]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-03-24T12:09:20.851 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:20 smithi090 ceph-mon[33646]: osdmap e61: 8 total, 8 up, 8 in 2024-03-24T12:09:21.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:20 smithi006 ceph-mon[41734]: Saving service rgw.foo.a spec with placement smithi006=foo.a;count:1 2024-03-24T12:09:21.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:20 smithi006 ceph-mon[41734]: osdmap e60: 8 total, 8 up, 8 in 2024-03-24T12:09:21.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:20 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/380256656' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-24T12:09:21.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:20 smithi006 ceph-mon[41734]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-24T12:09:21.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:20 smithi006 ceph-mon[41734]: pgmap v187: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-24T12:09:21.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:20 smithi006 ceph-mon[41734]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-03-24T12:09:21.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:20 smithi006 ceph-mon[41734]: osdmap e61: 8 total, 8 up, 8 in 2024-03-24T12:09:21.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:20 smithi006 ceph-mon[30108]: Saving service rgw.foo.a spec with placement smithi006=foo.a;count:1 2024-03-24T12:09:21.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:20 smithi006 ceph-mon[30108]: osdmap e60: 8 total, 8 up, 8 in 2024-03-24T12:09:21.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:20 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/380256656' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-24T12:09:21.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:20 smithi006 ceph-mon[30108]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-24T12:09:21.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:20 smithi006 ceph-mon[30108]: pgmap v187: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-24T12:09:21.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:20 smithi006 ceph-mon[30108]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-03-24T12:09:21.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:20 smithi006 ceph-mon[30108]: osdmap e61: 8 total, 8 up, 8 in 2024-03-24T12:09:22.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:22 smithi090 ceph-mon[33646]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:09:22.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:22 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:22.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:22 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:22.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:22 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:22.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:22 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:09:22.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:22 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:09:22.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.90:0/1225186493' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-24T12:09:22.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-24T12:09:22.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:22 smithi006 ceph-mon[30108]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:09:22.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:22 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:22.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:22 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:22.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:22 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:22.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:22 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:09:22.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:22 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:09:22.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.90:0/1225186493' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-24T12:09:22.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-24T12:09:22.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:22 smithi006 ceph-mon[41734]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:09:22.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:22 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:22.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:22 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:22.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:22 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:22.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:22 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:09:22.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:22 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:09:22.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.90:0/1225186493' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-24T12:09:22.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-24T12:09:23.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:23 smithi090 ceph-mon[33646]: Checking dashboard <-> RGW credentials 2024-03-24T12:09:23.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:23 smithi090 ceph-mon[33646]: pgmap v189: 36 pgs: 6 creating+activating, 17 active+clean, 13 unknown; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:09:23.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-03-24T12:09:23.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:23 smithi090 ceph-mon[33646]: osdmap e62: 8 total, 8 up, 8 in 2024-03-24T12:09:23.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-24T12:09:23.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/860734025' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-24T12:09:23.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:23 smithi090 ceph-mon[33646]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-24T12:09:23.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-24T12:09:23.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:23 smithi006 ceph-mon[41734]: Checking dashboard <-> RGW credentials 2024-03-24T12:09:23.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:23 smithi006 ceph-mon[41734]: pgmap v189: 36 pgs: 6 creating+activating, 17 active+clean, 13 unknown; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:09:23.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-03-24T12:09:23.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:23 smithi006 ceph-mon[41734]: osdmap e62: 8 total, 8 up, 8 in 2024-03-24T12:09:23.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-24T12:09:23.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/860734025' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-24T12:09:23.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:23 smithi006 ceph-mon[41734]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-24T12:09:23.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-24T12:09:23.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:23 smithi006 ceph-mon[30108]: Checking dashboard <-> RGW credentials 2024-03-24T12:09:23.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:23 smithi006 ceph-mon[30108]: pgmap v189: 36 pgs: 6 creating+activating, 17 active+clean, 13 unknown; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:09:23.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-03-24T12:09:23.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:23 smithi006 ceph-mon[30108]: osdmap e62: 8 total, 8 up, 8 in 2024-03-24T12:09:23.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-24T12:09:23.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/860734025' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-24T12:09:23.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:23 smithi006 ceph-mon[30108]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-24T12:09:23.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-24T12:09:24.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:24 smithi090 ceph-mon[33646]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-24T12:09:24.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:24 smithi090 ceph-mon[33646]: osdmap e63: 8 total, 8 up, 8 in 2024-03-24T12:09:24.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:24 smithi090 ceph-mon[33646]: pgmap v192: 68 pgs: 7 creating+peering, 6 creating+activating, 30 active+clean, 25 unknown; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 754 B/s wr, 3 op/s 2024-03-24T12:09:24.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:24 smithi006 ceph-mon[30108]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-24T12:09:24.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:24 smithi006 ceph-mon[30108]: osdmap e63: 8 total, 8 up, 8 in 2024-03-24T12:09:24.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:24 smithi006 ceph-mon[30108]: pgmap v192: 68 pgs: 7 creating+peering, 6 creating+activating, 30 active+clean, 25 unknown; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 754 B/s wr, 3 op/s 2024-03-24T12:09:24.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:24 smithi006 ceph-mon[41734]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-24T12:09:24.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:24 smithi006 ceph-mon[41734]: osdmap e63: 8 total, 8 up, 8 in 2024-03-24T12:09:24.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:24 smithi006 ceph-mon[41734]: pgmap v192: 68 pgs: 7 creating+peering, 6 creating+activating, 30 active+clean, 25 unknown; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 754 B/s wr, 3 op/s 2024-03-24T12:09:24.768 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.90 --placement '1;smithi090=iscsi.a' 2024-03-24T12:09:25.012 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T12:09:25.090 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:25 smithi090 ceph-mon[33646]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-24T12:09:25.090 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:25 smithi090 ceph-mon[33646]: Cluster is now healthy 2024-03-24T12:09:25.090 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:25 smithi090 ceph-mon[33646]: osdmap e64: 8 total, 8 up, 8 in 2024-03-24T12:09:25.090 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/860734025' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-24T12:09:25.090 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:25 smithi090 ceph-mon[33646]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-24T12:09:25.090 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-24T12:09:25.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:25 smithi006 ceph-mon[30108]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-24T12:09:25.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:25 smithi006 ceph-mon[30108]: Cluster is now healthy 2024-03-24T12:09:25.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:25 smithi006 ceph-mon[30108]: osdmap e64: 8 total, 8 up, 8 in 2024-03-24T12:09:25.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/860734025' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-24T12:09:25.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:25 smithi006 ceph-mon[30108]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-24T12:09:25.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-24T12:09:25.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:25 smithi006 ceph-mon[41734]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-24T12:09:25.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:25 smithi006 ceph-mon[41734]: Cluster is now healthy 2024-03-24T12:09:25.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:25 smithi006 ceph-mon[41734]: osdmap e64: 8 total, 8 up, 8 in 2024-03-24T12:09:25.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/860734025' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-24T12:09:25.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:25 smithi006 ceph-mon[41734]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-24T12:09:25.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-24T12:09:26.358 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:26 smithi090 ceph-mon[33646]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-24T12:09:26.358 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-24T12:09:26.359 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:26 smithi090 ceph-mon[33646]: osdmap e65: 8 total, 8 up, 8 in 2024-03-24T12:09:26.359 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:26 smithi090 ceph-mon[33646]: pgmap v195: 100 pgs: 7 creating+peering, 5 creating+activating, 39 active+clean, 49 unknown; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-24T12:09:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:26 smithi006 ceph-mon[30108]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-24T12:09:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-24T12:09:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:26 smithi006 ceph-mon[30108]: osdmap e65: 8 total, 8 up, 8 in 2024-03-24T12:09:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:26 smithi006 ceph-mon[30108]: pgmap v195: 100 pgs: 7 creating+peering, 5 creating+activating, 39 active+clean, 49 unknown; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-24T12:09:26.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:26 smithi006 ceph-mon[41734]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-24T12:09:26.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-24T12:09:26.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:26 smithi006 ceph-mon[41734]: osdmap e65: 8 total, 8 up, 8 in 2024-03-24T12:09:26.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:26 smithi006 ceph-mon[41734]: pgmap v195: 100 pgs: 7 creating+peering, 5 creating+activating, 39 active+clean, 49 unknown; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-24T12:09:26.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.575+0000 7fe9eb3f5700 1 -- 172.21.15.90:0/2752943527 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9e4102f90 msgr2=0x7fe9e41033f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:26.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.575+0000 7fe9eb3f5700 1 --2- 172.21.15.90:0/2752943527 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9e4102f90 0x7fe9e41033f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009990 tx=0x7fe9d4009ca0 comp rx=0 tx=0).stop 2024-03-24T12:09:26.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.576+0000 7fe9eb3f5700 1 -- 172.21.15.90:0/2752943527 shutdown_connections 2024-03-24T12:09:26.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.576+0000 7fe9eb3f5700 1 --2- 172.21.15.90:0/2752943527 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9e4103930 0x7fe9e410a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:26.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.576+0000 7fe9eb3f5700 1 --2- 172.21.15.90:0/2752943527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe9e40fffb0 0x7fe9e41003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:26.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.576+0000 7fe9eb3f5700 1 --2- 172.21.15.90:0/2752943527 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9e4102f90 0x7fe9e41033f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:26.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.576+0000 7fe9eb3f5700 1 -- 172.21.15.90:0/2752943527 >> 172.21.15.90:0/2752943527 conn(0x7fe9e40fb910 msgr2=0x7fe9e40fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:26.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.576+0000 7fe9eb3f5700 1 -- 172.21.15.90:0/2752943527 shutdown_connections 2024-03-24T12:09:26.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.576+0000 7fe9eb3f5700 1 -- 172.21.15.90:0/2752943527 wait complete. 2024-03-24T12:09:26.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.576+0000 7fe9eb3f5700 1 Processor -- start 2024-03-24T12:09:26.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.576+0000 7fe9eb3f5700 1 -- start start 2024-03-24T12:09:26.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.576+0000 7fe9eb3f5700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9e40fffb0 0x7fe9e4197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:26.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.576+0000 7fe9eb3f5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9e4102f90 0x7fe9e41975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:26.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.576+0000 7fe9eb3f5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe9e4103930 0x7fe9e419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:26.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.576+0000 7fe9eb3f5700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e419ec00 con 0x7fe9e4103930 2024-03-24T12:09:26.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.576+0000 7fe9eb3f5700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e419ed40 con 0x7fe9e4102f90 2024-03-24T12:09:26.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.576+0000 7fe9eb3f5700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e419eed0 con 0x7fe9e40fffb0 2024-03-24T12:09:26.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.576+0000 7fe9e8990700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9e4102f90 0x7fe9e41975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:26.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.576+0000 7fe9e8990700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9e4102f90 0x7fe9e41975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56448/0 (socket says 172.21.15.90:56448) 2024-03-24T12:09:26.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.577+0000 7fe9e8990700 1 -- 172.21.15.90:0/1414598124 learned_addr learned my addr 172.21.15.90:0/1414598124 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-24T12:09:26.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.577+0000 7fe9e9992700 1 --2- 172.21.15.90:0/1414598124 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe9e4103930 0x7fe9e419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:26.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.577+0000 7fe9e9191700 1 --2- 172.21.15.90:0/1414598124 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9e40fffb0 0x7fe9e4197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:26.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.577+0000 7fe9e8990700 1 -- 172.21.15.90:0/1414598124 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9e40fffb0 msgr2=0x7fe9e4197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:26.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.577+0000 7fe9e8990700 1 --2- 172.21.15.90:0/1414598124 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9e40fffb0 0x7fe9e4197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:26.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.577+0000 7fe9e8990700 1 -- 172.21.15.90:0/1414598124 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe9e4103930 msgr2=0x7fe9e419e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:26.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.577+0000 7fe9e8990700 1 --2- 172.21.15.90:0/1414598124 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe9e4103930 0x7fe9e419e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:26.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.577+0000 7fe9e8990700 1 -- 172.21.15.90:0/1414598124 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d40095f0 con 0x7fe9e4102f90 2024-03-24T12:09:26.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.577+0000 7fe9e8990700 1 --2- 172.21.15.90:0/1414598124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9e4102f90 0x7fe9e41975b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40098e0 tx=0x7fe9d4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:26.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.577+0000 7fe9da7fc700 1 -- 172.21.15.90:0/1414598124 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d401d070 con 0x7fe9e4102f90 2024-03-24T12:09:26.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.577+0000 7fe9eb3f5700 1 -- 172.21.15.90:0/1414598124 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e419f150 con 0x7fe9e4102f90 2024-03-24T12:09:26.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.577+0000 7fe9da7fc700 1 -- 172.21.15.90:0/1414598124 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe9d4022470 con 0x7fe9e4102f90 2024-03-24T12:09:26.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.577+0000 7fe9da7fc700 1 -- 172.21.15.90:0/1414598124 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d4003a20 con 0x7fe9e4102f90 2024-03-24T12:09:26.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.577+0000 7fe9eb3f5700 1 -- 172.21.15.90:0/1414598124 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e419f560 con 0x7fe9e4102f90 2024-03-24T12:09:26.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.578+0000 7fe9eb3f5700 1 -- 172.21.15.90:0/1414598124 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e4108970 con 0x7fe9e4102f90 2024-03-24T12:09:26.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.578+0000 7fe9da7fc700 1 -- 172.21.15.90:0/1414598124 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 15) v1 ==== 91079+0+0 (secure 0 0 0) 0x7fe9d400fb00 con 0x7fe9e4102f90 2024-03-24T12:09:26.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.579+0000 7fe9da7fc700 1 --2- 172.21.15.90:0/1414598124 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fe9d006d610 0x7fe9d006fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:26.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.579+0000 7fe9da7fc700 1 -- 172.21.15.90:0/1414598124 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7121+0+0 (secure 0 0 0) 0x7fe9d408e590 con 0x7fe9e4102f90 2024-03-24T12:09:26.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.579+0000 7fe9e9191700 1 --2- 172.21.15.90:0/1414598124 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fe9d006d610 0x7fe9d006fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:26.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.580+0000 7fe9e9191700 1 --2- 172.21.15.90:0/1414598124 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fe9d006d610 0x7fe9d006fad0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe9e406a190 tx=0x7fe9e000b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:26.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.584+0000 7fe9da7fc700 1 -- 172.21.15.90:0/1414598124 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe9d4057e80 con 0x7fe9e4102f90 2024-03-24T12:09:26.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.787+0000 7fe9eb3f5700 1 -- 172.21.15.90:0/1414598124 --> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.90", "placement": "1;smithi090=iscsi.a", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9e40620b0 con 0x7fe9d006d610 2024-03-24T12:09:26.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.796+0000 7fe9da7fc700 1 -- 172.21.15.90:0/1414598124 <== mgr.14152 v2:172.21.15.6:6800/3847865179 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+35 (secure 0 0 0) 0x7fe9e40620b0 con 0x7fe9d006d610 2024-03-24T12:09:26.795 INFO:teuthology.orchestra.run.smithi090.stdout:Scheduled iscsi.datapool update... 2024-03-24T12:09:26.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.798+0000 7fe9eb3f5700 1 -- 172.21.15.90:0/1414598124 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fe9d006d610 msgr2=0x7fe9d006fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:26.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.798+0000 7fe9eb3f5700 1 --2- 172.21.15.90:0/1414598124 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fe9d006d610 0x7fe9d006fad0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe9e406a190 tx=0x7fe9e000b5b0 comp rx=0 tx=0).stop 2024-03-24T12:09:26.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.798+0000 7fe9eb3f5700 1 -- 172.21.15.90:0/1414598124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9e4102f90 msgr2=0x7fe9e41975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:26.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.798+0000 7fe9eb3f5700 1 --2- 172.21.15.90:0/1414598124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9e4102f90 0x7fe9e41975b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40098e0 tx=0x7fe9d4009580 comp rx=0 tx=0).stop 2024-03-24T12:09:26.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.798+0000 7fe9eb3f5700 1 -- 172.21.15.90:0/1414598124 shutdown_connections 2024-03-24T12:09:26.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.798+0000 7fe9eb3f5700 1 --2- 172.21.15.90:0/1414598124 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7fe9d006d610 0x7fe9d006fad0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:26.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.798+0000 7fe9eb3f5700 1 --2- 172.21.15.90:0/1414598124 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe9e4103930 0x7fe9e419e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:26.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.798+0000 7fe9eb3f5700 1 --2- 172.21.15.90:0/1414598124 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9e40fffb0 0x7fe9e4197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:26.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.798+0000 7fe9eb3f5700 1 --2- 172.21.15.90:0/1414598124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9e4102f90 0x7fe9e41975b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:26.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.798+0000 7fe9eb3f5700 1 -- 172.21.15.90:0/1414598124 >> 172.21.15.90:0/1414598124 conn(0x7fe9e40fb910 msgr2=0x7fe9e40fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:26.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.798+0000 7fe9eb3f5700 1 -- 172.21.15.90:0/1414598124 shutdown_connections 2024-03-24T12:09:26.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:26.798+0000 7fe9eb3f5700 1 -- 172.21.15.90:0/1414598124 wait complete. 2024-03-24T12:09:27.370 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:27 smithi090 ceph-mon[33646]: osdmap e66: 8 total, 8 up, 8 in 2024-03-24T12:09:27.370 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:27 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/860734025' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-24T12:09:27.370 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:27 smithi090 ceph-mon[33646]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-24T12:09:27.370 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:27 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-24T12:09:27.370 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:27 smithi090 ceph-mon[33646]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.90", "placement": "1;smithi090=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:09:27.370 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:27 smithi090 ceph-mon[33646]: Saving service iscsi.datapool spec with placement smithi090=iscsi.a;count:1 2024-03-24T12:09:27.370 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:27 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:27.394 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-03-24T12:09:27.394 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-24T12:09:27.394 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-03-24T12:09:27.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:27 smithi006 ceph-mon[30108]: osdmap e66: 8 total, 8 up, 8 in 2024-03-24T12:09:27.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:27 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/860734025' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-24T12:09:27.424 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:27 smithi006 ceph-mon[30108]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-24T12:09:27.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:27 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-24T12:09:27.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:27 smithi006 ceph-mon[30108]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.90", "placement": "1;smithi090=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:09:27.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:27 smithi006 ceph-mon[30108]: Saving service iscsi.datapool spec with placement smithi090=iscsi.a;count:1 2024-03-24T12:09:27.425 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:27 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:27.425 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:27 smithi006 ceph-mon[41734]: osdmap e66: 8 total, 8 up, 8 in 2024-03-24T12:09:27.425 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:27 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/860734025' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-24T12:09:27.425 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:27 smithi006 ceph-mon[41734]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-24T12:09:27.425 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:27 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-24T12:09:27.425 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:27 smithi006 ceph-mon[41734]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.90", "placement": "1;smithi090=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:09:27.426 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:27 smithi006 ceph-mon[41734]: Saving service iscsi.datapool spec with placement smithi090=iscsi.a;count:1 2024-03-24T12:09:27.426 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:27 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:27.430 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-24T12:09:27.430 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-03-24T12:09:27.460 DEBUG:teuthology.orchestra.run.smithi090:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@iscsi.iscsi.a.service 2024-03-24T12:09:27.502 INFO:tasks.cephadm:Adding prometheus.a on smithi090 2024-03-24T12:09:27.502 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph orch apply prometheus '1;smithi090=a' 2024-03-24T12:09:27.784 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T12:09:28.266 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:28 smithi090 ceph-mon[33646]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-24T12:09:28.266 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-24T12:09:28.266 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:28 smithi090 ceph-mon[33646]: osdmap e67: 8 total, 8 up, 8 in 2024-03-24T12:09:28.266 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-24T12:09:28.266 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/860734025' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-24T12:09:28.266 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:28 smithi090 ceph-mon[33646]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-24T12:09:28.266 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:28 smithi090 ceph-mon[33646]: pgmap v198: 132 pgs: 22 creating+peering, 89 active+clean, 21 unknown; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-24T12:09:28.267 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-24T12:09:28.267 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:28 smithi090 ceph-mon[33646]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-24T12:09:28.267 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:28 smithi090 ceph-mon[33646]: osdmap e68: 8 total, 8 up, 8 in 2024-03-24T12:09:28.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:28 smithi006 ceph-mon[30108]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-24T12:09:28.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-24T12:09:28.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:28 smithi006 ceph-mon[30108]: osdmap e67: 8 total, 8 up, 8 in 2024-03-24T12:09:28.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-24T12:09:28.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/860734025' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-24T12:09:28.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:28 smithi006 ceph-mon[30108]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-24T12:09:28.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:28 smithi006 ceph-mon[30108]: pgmap v198: 132 pgs: 22 creating+peering, 89 active+clean, 21 unknown; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-24T12:09:28.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-24T12:09:28.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:28 smithi006 ceph-mon[30108]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-24T12:09:28.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:28 smithi006 ceph-mon[30108]: osdmap e68: 8 total, 8 up, 8 in 2024-03-24T12:09:28.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:28 smithi006 ceph-mon[41734]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-24T12:09:28.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-24T12:09:28.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:28 smithi006 ceph-mon[41734]: osdmap e67: 8 total, 8 up, 8 in 2024-03-24T12:09:28.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-24T12:09:28.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/860734025' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-24T12:09:28.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:28 smithi006 ceph-mon[41734]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-24T12:09:28.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:28 smithi006 ceph-mon[41734]: pgmap v198: 132 pgs: 22 creating+peering, 89 active+clean, 21 unknown; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-24T12:09:28.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/913383513' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-24T12:09:28.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:28 smithi006 ceph-mon[41734]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-24T12:09:28.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:28 smithi006 ceph-mon[41734]: osdmap e68: 8 total, 8 up, 8 in 2024-03-24T12:09:29.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.423+0000 7ff45604a700 1 -- 172.21.15.90:0/2672933380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff450105130 msgr2=0x7ff450107520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:29.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.423+0000 7ff45604a700 1 --2- 172.21.15.90:0/2672933380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff450105130 0x7ff450107520 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff444009990 tx=0x7ff444009ca0 comp rx=0 tx=0).stop 2024-03-24T12:09:29.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.423+0000 7ff45604a700 1 -- 172.21.15.90:0/2672933380 shutdown_connections 2024-03-24T12:09:29.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.423+0000 7ff45604a700 1 --2- 172.21.15.90:0/2672933380 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff450107a60 0x7ff450109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:29.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.423+0000 7ff45604a700 1 --2- 172.21.15.90:0/2672933380 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff450102800 0x7ff450104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:29.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.423+0000 7ff45604a700 1 --2- 172.21.15.90:0/2672933380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff450105130 0x7ff450107520 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:29.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.423+0000 7ff45604a700 1 -- 172.21.15.90:0/2672933380 >> 172.21.15.90:0/2672933380 conn(0x7ff4500fc1d0 msgr2=0x7ff4500fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:29.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.424+0000 7ff45604a700 1 -- 172.21.15.90:0/2672933380 shutdown_connections 2024-03-24T12:09:29.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.424+0000 7ff45604a700 1 -- 172.21.15.90:0/2672933380 wait complete. 2024-03-24T12:09:29.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.424+0000 7ff45604a700 1 Processor -- start 2024-03-24T12:09:29.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.424+0000 7ff45604a700 1 -- start start 2024-03-24T12:09:29.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.425+0000 7ff45604a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff450102800 0x7ff450197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:29.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.425+0000 7ff45604a700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff450105130 0x7ff450197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:29.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.425+0000 7ff45604a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff450107a60 0x7ff45019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:29.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.425+0000 7ff45604a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45019edc0 con 0x7ff450102800 2024-03-24T12:09:29.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.425+0000 7ff45604a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45019ef00 con 0x7ff450107a60 2024-03-24T12:09:29.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.425+0000 7ff45604a700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45019f090 con 0x7ff450105130 2024-03-24T12:09:29.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.425+0000 7ff44ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff450107a60 0x7ff45019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:29.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.425+0000 7ff44ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff450107a60 0x7ff45019e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56454/0 (socket says 172.21.15.90:56454) 2024-03-24T12:09:29.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.425+0000 7ff44ffff700 1 -- 172.21.15.90:0/3245968636 learned_addr learned my addr 172.21.15.90:0/3245968636 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-24T12:09:29.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.425+0000 7ff44f7fe700 1 --2- 172.21.15.90:0/3245968636 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff450102800 0x7ff450197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:29.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.425+0000 7ff44effd700 1 --2- 172.21.15.90:0/3245968636 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff450105130 0x7ff450197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:29.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.425+0000 7ff44ffff700 1 -- 172.21.15.90:0/3245968636 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff450105130 msgr2=0x7ff450197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:29.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.425+0000 7ff44ffff700 1 --2- 172.21.15.90:0/3245968636 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff450105130 0x7ff450197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:29.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.425+0000 7ff44ffff700 1 -- 172.21.15.90:0/3245968636 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff450102800 msgr2=0x7ff450197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:29.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.425+0000 7ff44ffff700 1 --2- 172.21.15.90:0/3245968636 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff450102800 0x7ff450197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:29.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.425+0000 7ff44ffff700 1 -- 172.21.15.90:0/3245968636 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4440095f0 con 0x7ff450107a60 2024-03-24T12:09:29.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.425+0000 7ff44ffff700 1 --2- 172.21.15.90:0/3245968636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff450107a60 0x7ff45019e7f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff44800b9e0 tx=0x7ff44800bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:29.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.425+0000 7ff44cff9700 1 -- 172.21.15.90:0/3245968636 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44800c9b0 con 0x7ff450107a60 2024-03-24T12:09:29.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.425+0000 7ff44cff9700 1 -- 172.21.15.90:0/3245968636 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ff448014440 con 0x7ff450107a60 2024-03-24T12:09:29.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.425+0000 7ff45604a700 1 -- 172.21.15.90:0/3245968636 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff45019f370 con 0x7ff450107a60 2024-03-24T12:09:29.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.425+0000 7ff44cff9700 1 -- 172.21.15.90:0/3245968636 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4480125a0 con 0x7ff450107a60 2024-03-24T12:09:29.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.425+0000 7ff45604a700 1 -- 172.21.15.90:0/3245968636 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff45019f820 con 0x7ff450107a60 2024-03-24T12:09:29.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.426+0000 7ff45604a700 1 -- 172.21.15.90:0/3245968636 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4500fdd50 con 0x7ff450107a60 2024-03-24T12:09:29.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.427+0000 7ff44cff9700 1 -- 172.21.15.90:0/3245968636 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 15) v1 ==== 91079+0+0 (secure 0 0 0) 0x7ff44800cb10 con 0x7ff450107a60 2024-03-24T12:09:29.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.427+0000 7ff44cff9700 1 --2- 172.21.15.90:0/3245968636 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ff43806d570 0x7ff43806fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:29.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.427+0000 7ff44cff9700 1 -- 172.21.15.90:0/3245968636 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7148+0+0 (secure 0 0 0) 0x7ff448014e40 con 0x7ff450107a60 2024-03-24T12:09:29.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.427+0000 7ff44f7fe700 1 --2- 172.21.15.90:0/3245968636 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ff43806d570 0x7ff43806fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:29.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.428+0000 7ff44f7fe700 1 --2- 172.21.15.90:0/3245968636 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ff43806d570 0x7ff43806fa30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff440005fd0 tx=0x7ff44000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:29.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.432+0000 7ff44cff9700 1 -- 172.21.15.90:0/3245968636 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff448056240 con 0x7ff450107a60 2024-03-24T12:09:29.540 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:29 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:29.540 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:29 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:29.540 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:29 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:29.540 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:29 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T12:09:29.540 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:29 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-03-24T12:09:29.540 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:29 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:09:29.540 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:29 smithi090 ceph-mon[33646]: Deploying daemon iscsi.iscsi.a on smithi090 2024-03-24T12:09:29.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.642+0000 7ff45604a700 1 -- 172.21.15.90:0/3245968636 --> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi090=a", "target": ["mon-mgr", ""]}) v1 -- 0x7ff45010cd00 con 0x7ff43806d570 2024-03-24T12:09:29.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.651+0000 7ff44cff9700 1 -- 172.21.15.90:0/3245968636 <== mgr.14152 v2:172.21.15.6:6800/3847865179 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7ff45010cd00 con 0x7ff43806d570 2024-03-24T12:09:29.650 INFO:teuthology.orchestra.run.smithi090.stdout:Scheduled prometheus update... 2024-03-24T12:09:29.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.653+0000 7ff45604a700 1 -- 172.21.15.90:0/3245968636 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ff43806d570 msgr2=0x7ff43806fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:29.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.653+0000 7ff45604a700 1 --2- 172.21.15.90:0/3245968636 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ff43806d570 0x7ff43806fa30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff440005fd0 tx=0x7ff44000a560 comp rx=0 tx=0).stop 2024-03-24T12:09:29.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.653+0000 7ff45604a700 1 -- 172.21.15.90:0/3245968636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff450107a60 msgr2=0x7ff45019e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:29.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.653+0000 7ff45604a700 1 --2- 172.21.15.90:0/3245968636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff450107a60 0x7ff45019e7f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff44800b9e0 tx=0x7ff44800bda0 comp rx=0 tx=0).stop 2024-03-24T12:09:29.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.653+0000 7ff45604a700 1 -- 172.21.15.90:0/3245968636 shutdown_connections 2024-03-24T12:09:29.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.653+0000 7ff45604a700 1 --2- 172.21.15.90:0/3245968636 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ff43806d570 0x7ff43806fa30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:29.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.653+0000 7ff45604a700 1 --2- 172.21.15.90:0/3245968636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff450107a60 0x7ff45019e7f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:29.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.653+0000 7ff45604a700 1 --2- 172.21.15.90:0/3245968636 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff450102800 0x7ff450197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:29.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.653+0000 7ff45604a700 1 --2- 172.21.15.90:0/3245968636 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff450105130 0x7ff450197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:29.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.654+0000 7ff45604a700 1 -- 172.21.15.90:0/3245968636 >> 172.21.15.90:0/3245968636 conn(0x7ff4500fc1d0 msgr2=0x7ff450103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:29.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.654+0000 7ff45604a700 1 -- 172.21.15.90:0/3245968636 shutdown_connections 2024-03-24T12:09:29.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:29.654+0000 7ff45604a700 1 -- 172.21.15.90:0/3245968636 wait complete. 2024-03-24T12:09:29.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:29 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:29.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:29 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:29.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:29 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:29.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:29 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T12:09:29.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:29 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-03-24T12:09:29.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:29 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:09:29.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:29 smithi006 ceph-mon[30108]: Deploying daemon iscsi.iscsi.a on smithi090 2024-03-24T12:09:29.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:29 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:29.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:29 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:29.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:29 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:29.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:29 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-24T12:09:29.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:29 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-03-24T12:09:29.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:29 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:09:29.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:29 smithi006 ceph-mon[41734]: Deploying daemon iscsi.iscsi.a on smithi090 2024-03-24T12:09:30.626 DEBUG:teuthology.orchestra.run.smithi090:prometheus.a> sudo journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@prometheus.a.service 2024-03-24T12:09:30.629 INFO:tasks.cephadm:Adding node-exporter.a on smithi006 2024-03-24T12:09:30.629 INFO:tasks.cephadm:Adding node-exporter.b on smithi090 2024-03-24T12:09:30.629 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph orch apply node-exporter '2;smithi006=a;smithi090=b' 2024-03-24T12:09:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:30 smithi090 ceph-mon[33646]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi090=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:09:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:30 smithi090 ceph-mon[33646]: Saving service prometheus spec with placement smithi090=a;count:1 2024-03-24T12:09:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:30 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:30 smithi090 ceph-mon[33646]: pgmap v200: 132 pgs: 15 creating+peering, 107 active+clean, 10 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-24T12:09:30.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:30 smithi006 ceph-mon[30108]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi090=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:09:30.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:30 smithi006 ceph-mon[30108]: Saving service prometheus spec with placement smithi090=a;count:1 2024-03-24T12:09:30.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:30 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:30.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:30 smithi006 ceph-mon[30108]: pgmap v200: 132 pgs: 15 creating+peering, 107 active+clean, 10 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-24T12:09:30.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:30 smithi006 ceph-mon[41734]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi090=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:09:30.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:30 smithi006 ceph-mon[41734]: Saving service prometheus spec with placement smithi090=a;count:1 2024-03-24T12:09:30.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:30 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:30.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:30 smithi006 ceph-mon[41734]: pgmap v200: 132 pgs: 15 creating+peering, 107 active+clean, 10 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-24T12:09:31.092 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T12:09:32.744 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:32 smithi090 ceph-mon[33646]: osdmap e69: 8 total, 8 up, 8 in 2024-03-24T12:09:32.744 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:32 smithi090 ceph-mon[33646]: pgmap v202: 132 pgs: 6 creating+peering, 126 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s rd, 5.2 KiB/s wr, 154 op/s 2024-03-24T12:09:32.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:32 smithi006 ceph-mon[30108]: osdmap e69: 8 total, 8 up, 8 in 2024-03-24T12:09:32.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:32 smithi006 ceph-mon[30108]: pgmap v202: 132 pgs: 6 creating+peering, 126 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s rd, 5.2 KiB/s wr, 154 op/s 2024-03-24T12:09:32.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:32 smithi006 ceph-mon[41734]: osdmap e69: 8 total, 8 up, 8 in 2024-03-24T12:09:32.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:32 smithi006 ceph-mon[41734]: pgmap v202: 132 pgs: 6 creating+peering, 126 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s rd, 5.2 KiB/s wr, 154 op/s 2024-03-24T12:09:33.344 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:09:33 smithi090 podman[62867]: 2024-03-24 12:09:33.167239844 +0000 UTC m=+0.343079644 container create 4184f9d18fedbd11681648594dc01a6bc78f9f232bd02a2acc1412491c7fea3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T12:09:33.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.449+0000 7f75fdf2d700 1 -- 172.21.15.90:0/1325711275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75f8102800 msgr2=0x7f75f8104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:33.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.449+0000 7f75fdf2d700 1 --2- 172.21.15.90:0/1325711275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75f8102800 0x7f75f8104bf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009990 tx=0x7f75e8009ca0 comp rx=0 tx=0).stop 2024-03-24T12:09:33.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.449+0000 7f75fdf2d700 1 -- 172.21.15.90:0/1325711275 shutdown_connections 2024-03-24T12:09:33.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.449+0000 7f75fdf2d700 1 --2- 172.21.15.90:0/1325711275 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75f8107a60 0x7f75f8109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:33.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.449+0000 7f75fdf2d700 1 --2- 172.21.15.90:0/1325711275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75f8102800 0x7f75f8104bf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:33.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.449+0000 7f75fdf2d700 1 --2- 172.21.15.90:0/1325711275 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75f8105130 0x7f75f8107520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:33.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.449+0000 7f75fdf2d700 1 -- 172.21.15.90:0/1325711275 >> 172.21.15.90:0/1325711275 conn(0x7f75f80fc1d0 msgr2=0x7f75f80fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:33.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.449+0000 7f75fdf2d700 1 -- 172.21.15.90:0/1325711275 shutdown_connections 2024-03-24T12:09:33.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.449+0000 7f75fdf2d700 1 -- 172.21.15.90:0/1325711275 wait complete. 2024-03-24T12:09:33.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.450+0000 7f75fdf2d700 1 Processor -- start 2024-03-24T12:09:33.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.450+0000 7f75fdf2d700 1 -- start start 2024-03-24T12:09:33.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.450+0000 7f75fdf2d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75f8102800 0x7f75f8197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:33.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.450+0000 7f75fdf2d700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75f8105130 0x7f75f8197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:33.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.450+0000 7f75fdf2d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75f8107a60 0x7f75f819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:33.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.450+0000 7f75fdf2d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f819ecd0 con 0x7f75f8102800 2024-03-24T12:09:33.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.450+0000 7f75fdf2d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f819ee10 con 0x7f75f8107a60 2024-03-24T12:09:33.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.450+0000 7f75fdf2d700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f819efa0 con 0x7f75f8105130 2024-03-24T12:09:33.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.450+0000 7f75f7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75f8107a60 0x7f75f819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:33.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.450+0000 7f75f7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75f8107a60 0x7f75f819e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56464/0 (socket says 172.21.15.90:56464) 2024-03-24T12:09:33.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.450+0000 7f75f7fff700 1 -- 172.21.15.90:0/1133678271 learned_addr learned my addr 172.21.15.90:0/1133678271 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-24T12:09:33.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.450+0000 7f75f6ffd700 1 --2- 172.21.15.90:0/1133678271 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75f8105130 0x7f75f8197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:33.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.450+0000 7f75f77fe700 1 --2- 172.21.15.90:0/1133678271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75f8102800 0x7f75f8197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:33.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.451+0000 7f75f7fff700 1 -- 172.21.15.90:0/1133678271 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75f8105130 msgr2=0x7f75f8197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:33.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.451+0000 7f75f7fff700 1 --2- 172.21.15.90:0/1133678271 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75f8105130 0x7f75f8197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:33.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.451+0000 7f75f7fff700 1 -- 172.21.15.90:0/1133678271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75f8102800 msgr2=0x7f75f8197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:33.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.451+0000 7f75f7fff700 1 --2- 172.21.15.90:0/1133678271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75f8102800 0x7f75f8197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:33.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.451+0000 7f75f7fff700 1 -- 172.21.15.90:0/1133678271 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e80095f0 con 0x7f75f8107a60 2024-03-24T12:09:33.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.451+0000 7f75f7fff700 1 --2- 172.21.15.90:0/1133678271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75f8107a60 0x7f75f819e700 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f75f00094f0 tx=0x7f75f000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:33.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.451+0000 7f75f4ff9700 1 -- 172.21.15.90:0/1133678271 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f0004db0 con 0x7f75f8107a60 2024-03-24T12:09:33.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.451+0000 7f75fdf2d700 1 -- 172.21.15.90:0/1133678271 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f819f280 con 0x7f75f8107a60 2024-03-24T12:09:33.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.451+0000 7f75f4ff9700 1 -- 172.21.15.90:0/1133678271 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f75f000f040 con 0x7f75f8107a60 2024-03-24T12:09:33.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.451+0000 7f75f4ff9700 1 -- 172.21.15.90:0/1133678271 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f0005870 con 0x7f75f8107a60 2024-03-24T12:09:33.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.451+0000 7f75fdf2d700 1 -- 172.21.15.90:0/1133678271 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f819f780 con 0x7f75f8107a60 2024-03-24T12:09:33.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.452+0000 7f75fdf2d700 1 -- 172.21.15.90:0/1133678271 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f80fdd50 con 0x7f75f8107a60 2024-03-24T12:09:33.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.453+0000 7f75f4ff9700 1 -- 172.21.15.90:0/1133678271 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 15) v1 ==== 91079+0+0 (secure 0 0 0) 0x7f75f000db20 con 0x7f75f8107a60 2024-03-24T12:09:33.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.453+0000 7f75f4ff9700 1 --2- 172.21.15.90:0/1133678271 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f75e006d570 0x7f75e006fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:33.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.453+0000 7f75f4ff9700 1 -- 172.21.15.90:0/1133678271 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f75f008c880 con 0x7f75f8107a60 2024-03-24T12:09:33.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.453+0000 7f75f77fe700 1 --2- 172.21.15.90:0/1133678271 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f75e006d570 0x7f75e006fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:33.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.454+0000 7f75f77fe700 1 --2- 172.21.15.90:0/1133678271 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f75e006d570 0x7f75e006fa30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f75e800b5c0 tx=0x7f75e8003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:33.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.458+0000 7f75f4ff9700 1 -- 172.21.15.90:0/1133678271 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f75f0056040 con 0x7f75f8107a60 2024-03-24T12:09:33.635 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:09:33 smithi090 podman[62867]: 2024-03-24 12:09:33.554424505 +0000 UTC m=+0.730264273 container init 4184f9d18fedbd11681648594dc01a6bc78f9f232bd02a2acc1412491c7fea3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-24T12:09:33.636 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:09:33 smithi090 podman[62867]: 2024-03-24 12:09:33.562361061 +0000 UTC m=+0.738200818 container start 4184f9d18fedbd11681648594dc01a6bc78f9f232bd02a2acc1412491c7fea3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-24T12:09:33.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.673+0000 7f75fdf2d700 1 -- 172.21.15.90:0/1133678271 --> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi006=a;smithi090=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f75f80620b0 con 0x7f75e006d570 2024-03-24T12:09:33.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.680+0000 7f75f4ff9700 1 -- 172.21.15.90:0/1133678271 <== mgr.14152 v2:172.21.15.6:6800/3847865179 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f75f80620b0 con 0x7f75e006d570 2024-03-24T12:09:33.680 INFO:teuthology.orchestra.run.smithi090.stdout:Scheduled node-exporter update... 2024-03-24T12:09:33.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.682+0000 7f75fdf2d700 1 -- 172.21.15.90:0/1133678271 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f75e006d570 msgr2=0x7f75e006fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:33.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.682+0000 7f75fdf2d700 1 --2- 172.21.15.90:0/1133678271 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f75e006d570 0x7f75e006fa30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f75e800b5c0 tx=0x7f75e8003680 comp rx=0 tx=0).stop 2024-03-24T12:09:33.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.682+0000 7f75fdf2d700 1 -- 172.21.15.90:0/1133678271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75f8107a60 msgr2=0x7f75f819e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:33.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.683+0000 7f75fdf2d700 1 --2- 172.21.15.90:0/1133678271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75f8107a60 0x7f75f819e700 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f75f00094f0 tx=0x7f75f000d3b0 comp rx=0 tx=0).stop 2024-03-24T12:09:33.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.683+0000 7f75fdf2d700 1 -- 172.21.15.90:0/1133678271 shutdown_connections 2024-03-24T12:09:33.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.683+0000 7f75fdf2d700 1 --2- 172.21.15.90:0/1133678271 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f75e006d570 0x7f75e006fa30 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f75e800b5c0 tx=0x7f75e8003680 comp rx=0 tx=0).stop 2024-03-24T12:09:33.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.683+0000 7f75fdf2d700 1 --2- 172.21.15.90:0/1133678271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75f8107a60 0x7f75f819e700 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:33.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.683+0000 7f75fdf2d700 1 --2- 172.21.15.90:0/1133678271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75f8102800 0x7f75f8197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:33.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.683+0000 7f75fdf2d700 1 --2- 172.21.15.90:0/1133678271 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75f8105130 0x7f75f8197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:33.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.683+0000 7f75fdf2d700 1 -- 172.21.15.90:0/1133678271 >> 172.21.15.90:0/1133678271 conn(0x7f75f80fc1d0 msgr2=0x7f75f8103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:33.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.683+0000 7f75fdf2d700 1 -- 172.21.15.90:0/1133678271 shutdown_connections 2024-03-24T12:09:33.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:33.683+0000 7f75fdf2d700 1 -- 172.21.15.90:0/1133678271 wait complete. 2024-03-24T12:09:33.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:09:33 smithi090 bash[62867]: 4184f9d18fedbd11681648594dc01a6bc78f9f232bd02a2acc1412491c7fea3d 2024-03-24T12:09:33.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:09:33 smithi090 systemd[1]: Started Ceph iscsi.iscsi.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:09:33.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:09:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug Started the configuration object watcher 2024-03-24T12:09:33.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:09:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug Checking for config object changes every 1s 2024-03-24T12:09:33.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:09:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug Processing osd blocklist entries for this node 2024-03-24T12:09:34.291 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:09:34 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug Reading the configuration object to update local LIO configuration 2024-03-24T12:09:34.292 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:09:34 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug Configuration does not have an entry for this host(smithi090.front.sepia.ceph.com) - nothing to define to LIO 2024-03-24T12:09:34.292 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:09:34 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug * Running on http://[::]:5000/ (Press CTRL+C to quit) 2024-03-24T12:09:34.326 DEBUG:teuthology.orchestra.run.smithi006:node-exporter.a> sudo journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@node-exporter.a.service 2024-03-24T12:09:34.328 DEBUG:teuthology.orchestra.run.smithi090:node-exporter.b> sudo journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@node-exporter.b.service 2024-03-24T12:09:34.331 INFO:tasks.cephadm:Adding alertmanager.a on smithi006 2024-03-24T12:09:34.331 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph orch apply alertmanager '1;smithi006=a' 2024-03-24T12:09:34.864 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T12:09:34.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:34 smithi006 ceph-mon[30108]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi006=a;smithi090=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:09:34.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:34 smithi006 ceph-mon[30108]: Saving service node-exporter spec with placement smithi006=a;smithi090=b;count:2 2024-03-24T12:09:34.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:34 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:34.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:34 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:34.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:34 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:34.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:34 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:34.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:34 smithi006 ceph-mon[30108]: Checking pool "datapool" exists for service iscsi.datapool 2024-03-24T12:09:34.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:34 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:34.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:34 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:09:34.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:34 smithi006 ceph-mon[30108]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s rd, 5.0 KiB/s wr, 164 op/s 2024-03-24T12:09:34.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.90:0/2985882156' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T12:09:34.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:34 smithi006 ceph-mon[41734]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi006=a;smithi090=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:09:34.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:34 smithi006 ceph-mon[41734]: Saving service node-exporter spec with placement smithi006=a;smithi090=b;count:2 2024-03-24T12:09:34.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:34 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:34.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:34 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:34.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:34 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:34.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:34 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:34.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:34 smithi006 ceph-mon[41734]: Checking pool "datapool" exists for service iscsi.datapool 2024-03-24T12:09:34.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:34 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:34.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:34 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:09:34.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:34 smithi006 ceph-mon[41734]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s rd, 5.0 KiB/s wr, 164 op/s 2024-03-24T12:09:34.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.90:0/2985882156' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T12:09:34.988 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:34 smithi090 ceph-mon[33646]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi006=a;smithi090=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:09:34.988 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:34 smithi090 ceph-mon[33646]: Saving service node-exporter spec with placement smithi006=a;smithi090=b;count:2 2024-03-24T12:09:34.988 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:34 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:34.989 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:34 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:34.989 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:34 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:34.989 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:34 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:34.989 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:34 smithi090 ceph-mon[33646]: Checking pool "datapool" exists for service iscsi.datapool 2024-03-24T12:09:34.989 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:34 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:34.989 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:34 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:09:34.989 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:34 smithi090 ceph-mon[33646]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 68 KiB/s rd, 5.0 KiB/s wr, 164 op/s 2024-03-24T12:09:34.989 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.90:0/2985882156' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-24T12:09:36.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:36 smithi090 ceph-mon[33646]: mgrmap e16: y(active, since 4m), standbys: x 2024-03-24T12:09:36.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:36 smithi090 ceph-mon[33646]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 139 op/s 2024-03-24T12:09:36.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:36 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:37.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:36 smithi006 ceph-mon[30108]: mgrmap e16: y(active, since 4m), standbys: x 2024-03-24T12:09:37.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:36 smithi006 ceph-mon[30108]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 139 op/s 2024-03-24T12:09:37.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:36 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:37.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:36 smithi006 ceph-mon[41734]: mgrmap e16: y(active, since 4m), standbys: x 2024-03-24T12:09:37.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:36 smithi006 ceph-mon[41734]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 139 op/s 2024-03-24T12:09:37.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:36 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:37.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.246+0000 7f1c4da07700 1 -- 172.21.15.90:0/3156850428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c481016e0 msgr2=0x7f1c48105f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:37.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.246+0000 7f1c4da07700 1 --2- 172.21.15.90:0/3156850428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c481016e0 0x7f1c48105f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000b0a0 tx=0x7f1c4000b3b0 comp rx=0 tx=0).stop 2024-03-24T12:09:37.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.246+0000 7f1c4da07700 1 -- 172.21.15.90:0/3156850428 shutdown_connections 2024-03-24T12:09:37.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.246+0000 7f1c4da07700 1 --2- 172.21.15.90:0/3156850428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c481016e0 0x7f1c48105f80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:37.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.246+0000 7f1c4da07700 1 --2- 172.21.15.90:0/3156850428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c480fffb0 0x7f1c481003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:37.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.246+0000 7f1c4da07700 1 --2- 172.21.15.90:0/3156850428 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c48100d20 0x7f1c481011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:37.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.246+0000 7f1c4da07700 1 -- 172.21.15.90:0/3156850428 >> 172.21.15.90:0/3156850428 conn(0x7f1c480fb910 msgr2=0x7f1c480fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:37.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.246+0000 7f1c4da07700 1 -- 172.21.15.90:0/3156850428 shutdown_connections 2024-03-24T12:09:37.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.246+0000 7f1c4da07700 1 -- 172.21.15.90:0/3156850428 wait complete. 2024-03-24T12:09:37.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.246+0000 7f1c4da07700 1 Processor -- start 2024-03-24T12:09:37.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.246+0000 7f1c4da07700 1 -- start start 2024-03-24T12:09:37.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.247+0000 7f1c4da07700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c480fffb0 0x7f1c4819d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:37.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.247+0000 7f1c4da07700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c48100d20 0x7f1c4819dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:37.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.247+0000 7f1c4da07700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c481016e0 0x7f1c481a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:37.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.247+0000 7f1c4da07700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c481a53a0 con 0x7f1c480fffb0 2024-03-24T12:09:37.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.247+0000 7f1c4da07700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c481a54e0 con 0x7f1c48100d20 2024-03-24T12:09:37.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.247+0000 7f1c4da07700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c481a5670 con 0x7f1c481016e0 2024-03-24T12:09:37.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.247+0000 7f1c467fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c48100d20 0x7f1c4819dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:37.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.247+0000 7f1c46ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c480fffb0 0x7f1c4819d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:37.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.247+0000 7f1c467fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c48100d20 0x7f1c4819dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50882/0 (socket says 172.21.15.90:50882) 2024-03-24T12:09:37.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.247+0000 7f1c467fc700 1 -- 172.21.15.90:0/3411373147 learned_addr learned my addr 172.21.15.90:0/3411373147 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-24T12:09:37.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.247+0000 7f1c477fe700 1 --2- 172.21.15.90:0/3411373147 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c481016e0 0x7f1c481a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:37.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.247+0000 7f1c467fc700 1 -- 172.21.15.90:0/3411373147 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c481016e0 msgr2=0x7f1c481a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:37.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.247+0000 7f1c467fc700 1 --2- 172.21.15.90:0/3411373147 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c481016e0 0x7f1c481a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:37.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.247+0000 7f1c467fc700 1 -- 172.21.15.90:0/3411373147 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c480fffb0 msgr2=0x7f1c4819d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:37.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.247+0000 7f1c467fc700 1 --2- 172.21.15.90:0/3411373147 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c480fffb0 0x7f1c4819d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:37.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.247+0000 7f1c467fc700 1 -- 172.21.15.90:0/3411373147 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c40009d00 con 0x7f1c48100d20 2024-03-24T12:09:37.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.247+0000 7f1c467fc700 1 --2- 172.21.15.90:0/3411373147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c48100d20 0x7f1c4819dd50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00c9b0 tx=0x7f1c3c00ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:37.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.247+0000 7f1c4ca05700 1 -- 172.21.15.90:0/3411373147 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c007b00 con 0x7f1c48100d20 2024-03-24T12:09:37.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.247+0000 7f1c4ca05700 1 -- 172.21.15.90:0/3411373147 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1c3c00ce50 con 0x7f1c48100d20 2024-03-24T12:09:37.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.247+0000 7f1c4ca05700 1 -- 172.21.15.90:0/3411373147 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c018400 con 0x7f1c48100d20 2024-03-24T12:09:37.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.247+0000 7f1c4da07700 1 -- 172.21.15.90:0/3411373147 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c481a5950 con 0x7f1c48100d20 2024-03-24T12:09:37.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.247+0000 7f1c4da07700 1 -- 172.21.15.90:0/3411373147 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c481a5e50 con 0x7f1c48100d20 2024-03-24T12:09:37.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.248+0000 7f1c2e7fc700 1 -- 172.21.15.90:0/3411373147 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c28005120 con 0x7f1c48100d20 2024-03-24T12:09:37.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.249+0000 7f1c4ca05700 1 -- 172.21.15.90:0/3411373147 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) v1 ==== 91130+0+0 (secure 0 0 0) 0x7f1c3c00f450 con 0x7f1c48100d20 2024-03-24T12:09:37.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.249+0000 7f1c4ca05700 1 --2- 172.21.15.90:0/3411373147 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f1c3006d600 0x7f1c3006fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:37.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.249+0000 7f1c4ca05700 1 -- 172.21.15.90:0/3411373147 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f1c3c08d0b0 con 0x7f1c48100d20 2024-03-24T12:09:37.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.249+0000 7f1c46ffd700 1 --2- 172.21.15.90:0/3411373147 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f1c3006d600 0x7f1c3006fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:37.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.250+0000 7f1c46ffd700 1 --2- 172.21.15.90:0/3411373147 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f1c3006d600 0x7f1c3006fac0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1c38005950 tx=0x7f1c3800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:37.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.254+0000 7f1c4ca05700 1 -- 172.21.15.90:0/3411373147 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1c3c056770 con 0x7f1c48100d20 2024-03-24T12:09:37.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.477+0000 7f1c2e7fc700 1 -- 172.21.15.90:0/3411373147 --> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi006=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c28000bc0 con 0x7f1c3006d600 2024-03-24T12:09:37.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.486+0000 7f1c4ca05700 1 -- 172.21.15.90:0/3411373147 <== mgr.14152 v2:172.21.15.6:6800/3847865179 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f1c28000bc0 con 0x7f1c3006d600 2024-03-24T12:09:37.485 INFO:teuthology.orchestra.run.smithi090.stdout:Scheduled alertmanager update... 2024-03-24T12:09:37.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.488+0000 7f1c2e7fc700 1 -- 172.21.15.90:0/3411373147 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f1c3006d600 msgr2=0x7f1c3006fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:37.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.488+0000 7f1c2e7fc700 1 --2- 172.21.15.90:0/3411373147 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f1c3006d600 0x7f1c3006fac0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1c38005950 tx=0x7f1c3800b410 comp rx=0 tx=0).stop 2024-03-24T12:09:37.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.488+0000 7f1c2e7fc700 1 -- 172.21.15.90:0/3411373147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c48100d20 msgr2=0x7f1c4819dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:37.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.488+0000 7f1c2e7fc700 1 --2- 172.21.15.90:0/3411373147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c48100d20 0x7f1c4819dd50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00c9b0 tx=0x7f1c3c00ccc0 comp rx=0 tx=0).stop 2024-03-24T12:09:37.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.488+0000 7f1c2e7fc700 1 -- 172.21.15.90:0/3411373147 shutdown_connections 2024-03-24T12:09:37.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.488+0000 7f1c2e7fc700 1 --2- 172.21.15.90:0/3411373147 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f1c3006d600 0x7f1c3006fac0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:37.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.488+0000 7f1c2e7fc700 1 --2- 172.21.15.90:0/3411373147 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c481016e0 0x7f1c481a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:37.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.488+0000 7f1c2e7fc700 1 --2- 172.21.15.90:0/3411373147 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c480fffb0 0x7f1c4819d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:37.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.488+0000 7f1c2e7fc700 1 --2- 172.21.15.90:0/3411373147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c48100d20 0x7f1c4819dd50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:37.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.488+0000 7f1c2e7fc700 1 -- 172.21.15.90:0/3411373147 >> 172.21.15.90:0/3411373147 conn(0x7f1c480fb910 msgr2=0x7f1c48101f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:37.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.488+0000 7f1c2e7fc700 1 -- 172.21.15.90:0/3411373147 shutdown_connections 2024-03-24T12:09:37.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:37.488+0000 7f1c2e7fc700 1 -- 172.21.15.90:0/3411373147 wait complete. 2024-03-24T12:09:38.299 DEBUG:teuthology.orchestra.run.smithi006:alertmanager.a> sudo journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@alertmanager.a.service 2024-03-24T12:09:38.301 INFO:tasks.cephadm:Adding grafana.a on smithi090 2024-03-24T12:09:38.301 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph orch apply grafana '1;smithi090=a' 2024-03-24T12:09:38.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:38 smithi090 ceph-mon[33646]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi006=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:09:38.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:38 smithi090 ceph-mon[33646]: Saving service alertmanager spec with placement smithi006=a;count:1 2024-03-24T12:09:38.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:38 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:38.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:38 smithi090 ceph-mon[33646]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 47 KiB/s rd, 3.6 KiB/s wr, 115 op/s 2024-03-24T12:09:38.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:38 smithi006 ceph-mon[30108]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi006=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:09:38.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:38 smithi006 ceph-mon[30108]: Saving service alertmanager spec with placement smithi006=a;count:1 2024-03-24T12:09:38.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:38 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:38.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:38 smithi006 ceph-mon[30108]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 47 KiB/s rd, 3.6 KiB/s wr, 115 op/s 2024-03-24T12:09:38.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:38 smithi006 ceph-mon[41734]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi006=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:09:38.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:38 smithi006 ceph-mon[41734]: Saving service alertmanager spec with placement smithi006=a;count:1 2024-03-24T12:09:38.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:38 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:38.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:38 smithi006 ceph-mon[41734]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 47 KiB/s rd, 3.6 KiB/s wr, 115 op/s 2024-03-24T12:09:38.720 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T12:09:39.741 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:39 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:39.741 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:39 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:39.741 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:39 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:09:39.741 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:39 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:09:39.741 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:39 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:39.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:39 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:39.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:39 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:39.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:39 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:09:39.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:39 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:09:39.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:39 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:39.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:39 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:39.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:39 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:39.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:39 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:09:39.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:39 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:09:39.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:39 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:40.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.266+0000 7ff8b1b66700 1 -- 172.21.15.90:0/1698250739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8ac102f60 msgr2=0x7ff8ac1033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:40.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.266+0000 7ff8b1b66700 1 --2- 172.21.15.90:0/1698250739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8ac102f60 0x7ff8ac1033c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0009990 tx=0x7ff8a0009ca0 comp rx=0 tx=0).stop 2024-03-24T12:09:40.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.267+0000 7ff8b1b66700 1 -- 172.21.15.90:0/1698250739 shutdown_connections 2024-03-24T12:09:40.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.267+0000 7ff8b1b66700 1 --2- 172.21.15.90:0/1698250739 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8ac103900 0x7ff8ac10a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:40.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.267+0000 7ff8b1b66700 1 --2- 172.21.15.90:0/1698250739 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff8ac0fff80 0x7ff8ac1003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:40.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.267+0000 7ff8b1b66700 1 --2- 172.21.15.90:0/1698250739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8ac102f60 0x7ff8ac1033c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:40.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.267+0000 7ff8b1b66700 1 -- 172.21.15.90:0/1698250739 >> 172.21.15.90:0/1698250739 conn(0x7ff8ac0fb900 msgr2=0x7ff8ac0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:40.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.267+0000 7ff8b1b66700 1 -- 172.21.15.90:0/1698250739 shutdown_connections 2024-03-24T12:09:40.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.267+0000 7ff8b1b66700 1 -- 172.21.15.90:0/1698250739 wait complete. 2024-03-24T12:09:40.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.267+0000 7ff8b1b66700 1 Processor -- start 2024-03-24T12:09:40.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.267+0000 7ff8b1b66700 1 -- start start 2024-03-24T12:09:40.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.268+0000 7ff8b1b66700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff8ac0fff80 0x7ff8ac197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:40.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.268+0000 7ff8b1b66700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8ac102f60 0x7ff8ac197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:40.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.268+0000 7ff8b1b66700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8ac103900 0x7ff8ac19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:40.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.268+0000 7ff8b1b66700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac19ede0 con 0x7ff8ac103900 2024-03-24T12:09:40.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.268+0000 7ff8b1b66700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac19ef20 con 0x7ff8ac102f60 2024-03-24T12:09:40.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.268+0000 7ff8b1b66700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac19f0b0 con 0x7ff8ac0fff80 2024-03-24T12:09:40.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.268+0000 7ff8aaffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8ac102f60 0x7ff8ac197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:40.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.268+0000 7ff8aaffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8ac102f60 0x7ff8ac197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50902/0 (socket says 172.21.15.90:50902) 2024-03-24T12:09:40.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.268+0000 7ff8aaffd700 1 -- 172.21.15.90:0/1166582513 learned_addr learned my addr 172.21.15.90:0/1166582513 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-24T12:09:40.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.268+0000 7ff8ab7fe700 1 --2- 172.21.15.90:0/1166582513 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff8ac0fff80 0x7ff8ac197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:40.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.268+0000 7ff8abfff700 1 --2- 172.21.15.90:0/1166582513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8ac103900 0x7ff8ac19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:40.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.268+0000 7ff8aaffd700 1 -- 172.21.15.90:0/1166582513 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff8ac0fff80 msgr2=0x7ff8ac197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:40.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.268+0000 7ff8aaffd700 1 --2- 172.21.15.90:0/1166582513 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff8ac0fff80 0x7ff8ac197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:40.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.268+0000 7ff8aaffd700 1 -- 172.21.15.90:0/1166582513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8ac103900 msgr2=0x7ff8ac19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:40.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.268+0000 7ff8aaffd700 1 --2- 172.21.15.90:0/1166582513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8ac103900 0x7ff8ac19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:40.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.268+0000 7ff8aaffd700 1 -- 172.21.15.90:0/1166582513 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a00095f0 con 0x7ff8ac102f60 2024-03-24T12:09:40.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.268+0000 7ff8abfff700 1 --2- 172.21.15.90:0/1166582513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8ac103900 0x7ff8ac19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:09:40.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.268+0000 7ff8aaffd700 1 --2- 172.21.15.90:0/1166582513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8ac102f60 0x7ff8ac197790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00098e0 tx=0x7ff8a000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:40.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.268+0000 7ff8a8ff9700 1 -- 172.21.15.90:0/1166582513 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a001d070 con 0x7ff8ac102f60 2024-03-24T12:09:40.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.268+0000 7ff8a8ff9700 1 -- 172.21.15.90:0/1166582513 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ff8a000fe90 con 0x7ff8ac102f60 2024-03-24T12:09:40.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.268+0000 7ff8b1b66700 1 -- 172.21.15.90:0/1166582513 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8ac19f330 con 0x7ff8ac102f60 2024-03-24T12:09:40.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.268+0000 7ff8a8ff9700 1 -- 172.21.15.90:0/1166582513 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0022df0 con 0x7ff8ac102f60 2024-03-24T12:09:40.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.269+0000 7ff8b1b66700 1 -- 172.21.15.90:0/1166582513 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ac19f740 con 0x7ff8ac102f60 2024-03-24T12:09:40.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.270+0000 7ff8b1b66700 1 -- 172.21.15.90:0/1166582513 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8ac108940 con 0x7ff8ac102f60 2024-03-24T12:09:40.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.270+0000 7ff8a8ff9700 1 -- 172.21.15.90:0/1166582513 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) v1 ==== 91130+0+0 (secure 0 0 0) 0x7ff8a000fa10 con 0x7ff8ac102f60 2024-03-24T12:09:40.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.270+0000 7ff8a8ff9700 1 --2- 172.21.15.90:0/1166582513 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ff89406d600 0x7ff89406fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:40.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.270+0000 7ff8a8ff9700 1 -- 172.21.15.90:0/1166582513 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7148+0+0 (secure 0 0 0) 0x7ff8a008e740 con 0x7ff8ac102f60 2024-03-24T12:09:40.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.271+0000 7ff8ab7fe700 1 --2- 172.21.15.90:0/1166582513 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ff89406d600 0x7ff89406fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:40.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.271+0000 7ff8ab7fe700 1 --2- 172.21.15.90:0/1166582513 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ff89406d600 0x7ff89406fac0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac06a1d0 tx=0x7ff89c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:40.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.275+0000 7ff8a8ff9700 1 -- 172.21.15.90:0/1166582513 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff8a0057fd0 con 0x7ff8ac102f60 2024-03-24T12:09:40.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.501+0000 7ff8b1b66700 1 -- 172.21.15.90:0/1166582513 --> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi090=a", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8ac0620b0 con 0x7ff89406d600 2024-03-24T12:09:40.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.510+0000 7ff8a8ff9700 1 -- 172.21.15.90:0/1166582513 <== mgr.14152 v2:172.21.15.6:6800/3847865179 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7ff8ac0620b0 con 0x7ff89406d600 2024-03-24T12:09:40.510 INFO:teuthology.orchestra.run.smithi090.stdout:Scheduled grafana update... 2024-03-24T12:09:40.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.512+0000 7ff8b1b66700 1 -- 172.21.15.90:0/1166582513 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ff89406d600 msgr2=0x7ff89406fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:40.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.512+0000 7ff8b1b66700 1 --2- 172.21.15.90:0/1166582513 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ff89406d600 0x7ff89406fac0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac06a1d0 tx=0x7ff89c009190 comp rx=0 tx=0).stop 2024-03-24T12:09:40.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.512+0000 7ff8b1b66700 1 -- 172.21.15.90:0/1166582513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8ac102f60 msgr2=0x7ff8ac197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:40.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.512+0000 7ff8b1b66700 1 --2- 172.21.15.90:0/1166582513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8ac102f60 0x7ff8ac197790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00098e0 tx=0x7ff8a000f690 comp rx=0 tx=0).stop 2024-03-24T12:09:40.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.512+0000 7ff8b1b66700 1 -- 172.21.15.90:0/1166582513 shutdown_connections 2024-03-24T12:09:40.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.512+0000 7ff8b1b66700 1 --2- 172.21.15.90:0/1166582513 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ff89406d600 0x7ff89406fac0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:40.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.512+0000 7ff8b1b66700 1 --2- 172.21.15.90:0/1166582513 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8ac103900 0x7ff8ac19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:40.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.512+0000 7ff8b1b66700 1 --2- 172.21.15.90:0/1166582513 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff8ac0fff80 0x7ff8ac197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:40.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.512+0000 7ff8b1b66700 1 --2- 172.21.15.90:0/1166582513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8ac102f60 0x7ff8ac197790 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:40.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.512+0000 7ff8b1b66700 1 -- 172.21.15.90:0/1166582513 >> 172.21.15.90:0/1166582513 conn(0x7ff8ac0fb900 msgr2=0x7ff8ac0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:40.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.512+0000 7ff8b1b66700 1 -- 172.21.15.90:0/1166582513 shutdown_connections 2024-03-24T12:09:40.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:40.512+0000 7ff8b1b66700 1 -- 172.21.15.90:0/1166582513 wait complete. 2024-03-24T12:09:41.101 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:40 smithi090 ceph-mon[33646]: Deploying daemon prometheus.a on smithi090 2024-03-24T12:09:41.102 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:40 smithi090 ceph-mon[33646]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-03-24T12:09:41.102 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:40 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:41.130 DEBUG:teuthology.orchestra.run.smithi090:grafana.a> sudo journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@grafana.a.service 2024-03-24T12:09:41.132 INFO:tasks.cephadm:Setting up client nodes... 2024-03-24T12:09:41.132 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-24T12:09:41.160 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:40 smithi006 ceph-mon[30108]: Deploying daemon prometheus.a on smithi090 2024-03-24T12:09:41.161 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:40 smithi006 ceph-mon[30108]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-03-24T12:09:41.161 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:40 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:41.161 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:40 smithi006 ceph-mon[41734]: Deploying daemon prometheus.a on smithi090 2024-03-24T12:09:41.161 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:40 smithi006 ceph-mon[41734]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-03-24T12:09:41.161 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:40 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:41.387 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:09:41.760 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:41 smithi006 ceph-mon[30108]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi090=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:09:41.760 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:41 smithi006 ceph-mon[30108]: Saving service grafana spec with placement smithi090=a;count:1 2024-03-24T12:09:41.760 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:41 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:41.760 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:41 smithi006 ceph-mon[41734]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi090=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:09:41.760 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:41 smithi006 ceph-mon[41734]: Saving service grafana spec with placement smithi090=a;count:1 2024-03-24T12:09:41.760 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:41 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:42.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:41 smithi090 ceph-mon[33646]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi090=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-24T12:09:42.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:41 smithi090 ceph-mon[33646]: Saving service grafana spec with placement smithi090=a;count:1 2024-03-24T12:09:42.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:41 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:42.804 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:42 smithi006 ceph-mon[30108]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 981 B/s wr, 27 op/s 2024-03-24T12:09:42.804 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:42 smithi006 ceph-mon[41734]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 981 B/s wr, 27 op/s 2024-03-24T12:09:43.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.041+0000 7f07ffd78700 1 -- 172.21.15.6:0/4292955612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07f8102760 msgr2=0x7f07f8109010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:43.040 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.041+0000 7f07ffd78700 1 --2- 172.21.15.6:0/4292955612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07f8102760 0x7f07f8109010 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f07e8009990 tx=0x7f07e8009ca0 comp rx=0 tx=0).stop 2024-03-24T12:09:43.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.041+0000 7f07ffd78700 1 -- 172.21.15.6:0/4292955612 shutdown_connections 2024-03-24T12:09:43.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.041+0000 7f07ffd78700 1 --2- 172.21.15.6:0/4292955612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07f8102760 0x7f07f8109010 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:43.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.041+0000 7f07ffd78700 1 --2- 172.21.15.6:0/4292955612 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f07f8100bc0 0x7f07f8100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:43.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.041+0000 7f07ffd78700 1 --2- 172.21.15.6:0/4292955612 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07f8101dc0 0x7f07f8102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:43.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.041+0000 7f07ffd78700 1 -- 172.21.15.6:0/4292955612 >> 172.21.15.6:0/4292955612 conn(0x7f07f80fc160 msgr2=0x7f07f80fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:43.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.041+0000 7f07ffd78700 1 -- 172.21.15.6:0/4292955612 shutdown_connections 2024-03-24T12:09:43.041 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.041+0000 7f07ffd78700 1 -- 172.21.15.6:0/4292955612 wait complete. 2024-03-24T12:09:43.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.041+0000 7f07ffd78700 1 Processor -- start 2024-03-24T12:09:43.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.042+0000 7f07ffd78700 1 -- start start 2024-03-24T12:09:43.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.042+0000 7f07ffd78700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f07f8100bc0 0x7f07f81970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:43.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.042+0000 7f07ffd78700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07f8101dc0 0x7f07f81975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:43.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.042+0000 7f07ffd78700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07f8102760 0x7f07f819e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:43.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.042+0000 7f07ffd78700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f819ec30 con 0x7f07f8102760 2024-03-24T12:09:43.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.042+0000 7f07ffd78700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f819ed70 con 0x7f07f8101dc0 2024-03-24T12:09:43.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.042+0000 7f07ffd78700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f819ef00 con 0x7f07f8100bc0 2024-03-24T12:09:43.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.042+0000 7f07fdb14700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f07f8100bc0 0x7f07f81970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:43.043 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.042+0000 7f07fdb14700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f07f8100bc0 0x7f07f81970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:33136/0 (socket says 172.21.15.6:33136) 2024-03-24T12:09:43.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.042+0000 7f07fdb14700 1 -- 172.21.15.6:0/755516459 learned_addr learned my addr 172.21.15.6:0/755516459 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:09:43.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.042+0000 7f07fd313700 1 --2- 172.21.15.6:0/755516459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07f8101dc0 0x7f07f81975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:43.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.042+0000 7f07fe315700 1 --2- 172.21.15.6:0/755516459 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07f8102760 0x7f07f819e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:43.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.042+0000 7f07fdb14700 1 -- 172.21.15.6:0/755516459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07f8101dc0 msgr2=0x7f07f81975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:43.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.042+0000 7f07fdb14700 1 --2- 172.21.15.6:0/755516459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07f8101dc0 0x7f07f81975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:43.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.042+0000 7f07fdb14700 1 -- 172.21.15.6:0/755516459 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07f8102760 msgr2=0x7f07f819e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:43.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.042+0000 7f07fdb14700 1 --2- 172.21.15.6:0/755516459 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07f8102760 0x7f07f819e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:43.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.042+0000 7f07fdb14700 1 -- 172.21.15.6:0/755516459 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f0009c40 con 0x7f07f8100bc0 2024-03-24T12:09:43.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.042+0000 7f07fd313700 1 --2- 172.21.15.6:0/755516459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07f8101dc0 0x7f07f81975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:09:43.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.042+0000 7f07fdb14700 1 --2- 172.21.15.6:0/755516459 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f07f8100bc0 0x7f07f81970a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f07f000c170 tx=0x7f07f000c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:43.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.042+0000 7f07f6ffd700 1 -- 172.21.15.6:0/755516459 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f000e050 con 0x7f07f8100bc0 2024-03-24T12:09:43.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.043+0000 7f07ffd78700 1 -- 172.21.15.6:0/755516459 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e80095f0 con 0x7f07f8100bc0 2024-03-24T12:09:43.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.043+0000 7f07f6ffd700 1 -- 172.21.15.6:0/755516459 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f07f000f040 con 0x7f07f8100bc0 2024-03-24T12:09:43.044 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.043+0000 7f07f6ffd700 1 -- 172.21.15.6:0/755516459 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0007e70 con 0x7f07f8100bc0 2024-03-24T12:09:43.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.043+0000 7f07ffd78700 1 -- 172.21.15.6:0/755516459 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f819f540 con 0x7f07f8100bc0 2024-03-24T12:09:43.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.044+0000 7f07ffd78700 1 -- 172.21.15.6:0/755516459 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f8107f80 con 0x7f07f8100bc0 2024-03-24T12:09:43.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.044+0000 7f07f6ffd700 1 -- 172.21.15.6:0/755516459 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 16) v1 ==== 91130+0+0 (secure 0 0 0) 0x7f07f0007500 con 0x7f07f8100bc0 2024-03-24T12:09:43.045 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.044+0000 7f07f6ffd700 1 --2- 172.21.15.6:0/755516459 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f07e006d6b0 0x7f07e006fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:43.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.044+0000 7f07f6ffd700 1 -- 172.21.15.6:0/755516459 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f07f008d070 con 0x7f07f8100bc0 2024-03-24T12:09:43.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.044+0000 7f07fd313700 1 --2- 172.21.15.6:0/755516459 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f07e006d6b0 0x7f07e006fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:43.046 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.045+0000 7f07fd313700 1 --2- 172.21.15.6:0/755516459 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f07e006d6b0 0x7f07e006fb70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f07ec011490 tx=0x7f07ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:43.048 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.049+0000 7f07f6ffd700 1 -- 172.21.15.6:0/755516459 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f07f0056730 con 0x7f07f8100bc0 2024-03-24T12:09:43.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:42 smithi090 ceph-mon[33646]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 981 B/s wr, 27 op/s 2024-03-24T12:09:43.335 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.336+0000 7f07ffd78700 1 -- 172.21.15.6:0/755516459 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f07f810cd80 con 0x7f07f8100bc0 2024-03-24T12:09:43.340 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.340+0000 7f07f6ffd700 1 -- 172.21.15.6:0/755516459 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7f07f0059df0 con 0x7f07f8100bc0 2024-03-24T12:09:43.340 INFO:teuthology.orchestra.run.smithi006.stdout:[client.0] 2024-03-24T12:09:43.340 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQAHGABm6VYnFBAAlK/kZP9dNO3rTobkUuAUog== 2024-03-24T12:09:43.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.343+0000 7f07ffd78700 1 -- 172.21.15.6:0/755516459 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f07e006d6b0 msgr2=0x7f07e006fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:43.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.343+0000 7f07ffd78700 1 --2- 172.21.15.6:0/755516459 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f07e006d6b0 0x7f07e006fb70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f07ec011490 tx=0x7f07ec009260 comp rx=0 tx=0).stop 2024-03-24T12:09:43.342 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.343+0000 7f07ffd78700 1 -- 172.21.15.6:0/755516459 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f07f8100bc0 msgr2=0x7f07f81970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:43.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.343+0000 7f07ffd78700 1 --2- 172.21.15.6:0/755516459 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f07f8100bc0 0x7f07f81970a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f07f000c170 tx=0x7f07f000c480 comp rx=0 tx=0).stop 2024-03-24T12:09:43.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.343+0000 7f07ffd78700 1 -- 172.21.15.6:0/755516459 shutdown_connections 2024-03-24T12:09:43.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.343+0000 7f07ffd78700 1 --2- 172.21.15.6:0/755516459 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f07e006d6b0 0x7f07e006fb70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:43.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.343+0000 7f07ffd78700 1 --2- 172.21.15.6:0/755516459 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07f8102760 0x7f07f819e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:43.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.343+0000 7f07ffd78700 1 --2- 172.21.15.6:0/755516459 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f07f8100bc0 0x7f07f81970a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:43.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.343+0000 7f07ffd78700 1 --2- 172.21.15.6:0/755516459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07f8101dc0 0x7f07f81975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:43.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.343+0000 7f07ffd78700 1 -- 172.21.15.6:0/755516459 >> 172.21.15.6:0/755516459 conn(0x7f07f80fc160 msgr2=0x7f07f81050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:43.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.343+0000 7f07ffd78700 1 -- 172.21.15.6:0/755516459 shutdown_connections 2024-03-24T12:09:43.343 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:43.343+0000 7f07ffd78700 1 -- 172.21.15.6:0/755516459 wait complete. 2024-03-24T12:09:43.937 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:43 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/755516459' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T12:09:43.937 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:43 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T12:09:43.937 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:43 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-24T12:09:43.937 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:43 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/755516459' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T12:09:43.938 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:43 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T12:09:43.938 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:43 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-24T12:09:43.967 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-24T12:09:43.967 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-24T12:09:43.967 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-24T12:09:44.053 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-24T12:09:44.077 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:43 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/755516459' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T12:09:44.078 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:43 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T12:09:44.078 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:43 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-24T12:09:44.078 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:09:43 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:09:44.287 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.b/config 2024-03-24T12:09:45.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:44 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:09:45.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:44 smithi090 ceph-mon[33646]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 8.7 KiB/s rd, 767 B/s wr, 20 op/s 2024-03-24T12:09:45.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:44 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:09:45.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:44 smithi006 ceph-mon[30108]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 8.7 KiB/s rd, 767 B/s wr, 20 op/s 2024-03-24T12:09:45.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:44 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:09:45.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:44 smithi006 ceph-mon[41734]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 8.7 KiB/s rd, 767 B/s wr, 20 op/s 2024-03-24T12:09:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-03-24T12:09:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2024-03-24T12:09:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:45 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2024-03-24T12:09:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:45 smithi006 ceph-mon[30108]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s 2024-03-24T12:09:46.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:45 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-03-24T12:09:46.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:45 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2024-03-24T12:09:46.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:45 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2024-03-24T12:09:46.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:45 smithi006 ceph-mon[41734]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s 2024-03-24T12:09:46.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:45 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-03-24T12:09:46.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:45 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2024-03-24T12:09:46.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:45 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2024-03-24T12:09:46.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:45 smithi090 ceph-mon[33646]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s 2024-03-24T12:09:47.200 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:46 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-03-24T12:09:47.200 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:46 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]': finished 2024-03-24T12:09:47.200 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:46 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]': finished 2024-03-24T12:09:47.200 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:46 smithi090 ceph-mon[33646]: osdmap e70: 8 total, 8 up, 8 in 2024-03-24T12:09:47.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:46 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-03-24T12:09:47.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:46 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]': finished 2024-03-24T12:09:47.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:46 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]': finished 2024-03-24T12:09:47.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:46 smithi006 ceph-mon[30108]: osdmap e70: 8 total, 8 up, 8 in 2024-03-24T12:09:47.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:46 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-03-24T12:09:47.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:46 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]': finished 2024-03-24T12:09:47.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:46 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]': finished 2024-03-24T12:09:47.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:46 smithi006 ceph-mon[41734]: osdmap e70: 8 total, 8 up, 8 in 2024-03-24T12:09:47.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.749+0000 7f5522124700 1 -- 172.21.15.90:0/129975446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f551c102760 msgr2=0x7f551c104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:47.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.749+0000 7f5522124700 1 --2- 172.21.15.90:0/129975446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f551c102760 0x7f551c104b50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f550c009990 tx=0x7f550c009ca0 comp rx=0 tx=0).stop 2024-03-24T12:09:47.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.749+0000 7f5522124700 1 -- 172.21.15.90:0/129975446 shutdown_connections 2024-03-24T12:09:47.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.749+0000 7f5522124700 1 --2- 172.21.15.90:0/129975446 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f551c1079c0 0x7f551c109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:47.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.749+0000 7f5522124700 1 --2- 172.21.15.90:0/129975446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f551c102760 0x7f551c104b50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:47.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.749+0000 7f5522124700 1 --2- 172.21.15.90:0/129975446 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f551c105090 0x7f551c107480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:47.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.749+0000 7f5522124700 1 -- 172.21.15.90:0/129975446 >> 172.21.15.90:0/129975446 conn(0x7f551c0fc190 msgr2=0x7f551c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:47.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.749+0000 7f5522124700 1 -- 172.21.15.90:0/129975446 shutdown_connections 2024-03-24T12:09:47.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.750+0000 7f5522124700 1 -- 172.21.15.90:0/129975446 wait complete. 2024-03-24T12:09:47.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.750+0000 7f5522124700 1 Processor -- start 2024-03-24T12:09:47.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.750+0000 7f5522124700 1 -- start start 2024-03-24T12:09:47.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.751+0000 7f5522124700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f551c102760 0x7f551c1971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:47.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.751+0000 7f5522124700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f551c105090 0x7f551c1976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:47.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.751+0000 7f5522124700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f551c1079c0 0x7f551c19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:47.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.751+0000 7f5522124700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c19ecf0 con 0x7f551c105090 2024-03-24T12:09:47.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.751+0000 7f5522124700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c19ee30 con 0x7f551c1079c0 2024-03-24T12:09:47.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.751+0000 7f5522124700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551c19efc0 con 0x7f551c102760 2024-03-24T12:09:47.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.751+0000 7f551bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f551c1079c0 0x7f551c19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:47.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.751+0000 7f551bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f551c1079c0 0x7f551c19e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60262/0 (socket says 172.21.15.90:60262) 2024-03-24T12:09:47.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.751+0000 7f551bfff700 1 -- 172.21.15.90:0/3788305781 learned_addr learned my addr 172.21.15.90:0/3788305781 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-24T12:09:47.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.751+0000 7f551affd700 1 --2- 172.21.15.90:0/3788305781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f551c105090 0x7f551c1976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:47.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.751+0000 7f551b7fe700 1 --2- 172.21.15.90:0/3788305781 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f551c102760 0x7f551c1971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:47.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.751+0000 7f551bfff700 1 -- 172.21.15.90:0/3788305781 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f551c102760 msgr2=0x7f551c1971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:47.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.751+0000 7f551bfff700 1 --2- 172.21.15.90:0/3788305781 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f551c102760 0x7f551c1971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:47.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.751+0000 7f551bfff700 1 -- 172.21.15.90:0/3788305781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f551c105090 msgr2=0x7f551c1976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:47.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.751+0000 7f551bfff700 1 --2- 172.21.15.90:0/3788305781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f551c105090 0x7f551c1976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:47.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.751+0000 7f551bfff700 1 -- 172.21.15.90:0/3788305781 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550c0095f0 con 0x7f551c1079c0 2024-03-24T12:09:47.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.751+0000 7f551bfff700 1 --2- 172.21.15.90:0/3788305781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f551c1079c0 0x7f551c19e720 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f551400b9e0 tx=0x7f551400bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:47.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.751+0000 7f5518ff9700 1 -- 172.21.15.90:0/3788305781 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551400d8e0 con 0x7f551c1079c0 2024-03-24T12:09:47.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.751+0000 7f5522124700 1 -- 172.21.15.90:0/3788305781 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f551c19f2a0 con 0x7f551c1079c0 2024-03-24T12:09:47.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.751+0000 7f5518ff9700 1 -- 172.21.15.90:0/3788305781 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f5514009880 con 0x7f551c1079c0 2024-03-24T12:09:47.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.751+0000 7f5518ff9700 1 -- 172.21.15.90:0/3788305781 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514015e40 con 0x7f551c1079c0 2024-03-24T12:09:47.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.751+0000 7f5522124700 1 -- 172.21.15.90:0/3788305781 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551c19f750 con 0x7f551c1079c0 2024-03-24T12:09:47.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.752+0000 7f5522124700 1 -- 172.21.15.90:0/3788305781 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f551c0fdd10 con 0x7f551c1079c0 2024-03-24T12:09:47.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.753+0000 7f5518ff9700 1 -- 172.21.15.90:0/3788305781 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) v1 ==== 91130+0+0 (secure 0 0 0) 0x7f551400da40 con 0x7f551c1079c0 2024-03-24T12:09:47.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.753+0000 7f5518ff9700 1 --2- 172.21.15.90:0/3788305781 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f550406d6b0 0x7f550406fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:47.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.753+0000 7f5518ff9700 1 -- 172.21.15.90:0/3788305781 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7207+0+0 (secure 0 0 0) 0x7f5514011070 con 0x7f551c1079c0 2024-03-24T12:09:47.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.754+0000 7f551b7fe700 1 --2- 172.21.15.90:0/3788305781 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f550406d6b0 0x7f550406fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:47.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.754+0000 7f551b7fe700 1 --2- 172.21.15.90:0/3788305781 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f550406d6b0 0x7f550406fb70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f550c00b5c0 tx=0x7f550c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:47.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:47.758+0000 7f5518ff9700 1 -- 172.21.15.90:0/3788305781 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f551404fd20 con 0x7f551c1079c0 2024-03-24T12:09:48.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:48.035+0000 7f5522124700 1 -- 172.21.15.90:0/3788305781 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f551c10ce20 con 0x7f551c1079c0 2024-03-24T12:09:48.034 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:47 smithi090 ceph-mon[33646]: osdmap e71: 8 total, 8 up, 8 in 2024-03-24T12:09:48.035 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:47 smithi090 ceph-mon[33646]: pgmap v212: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:09:48.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:48.041+0000 7f5518ff9700 1 -- 172.21.15.90:0/3788305781 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f5514056170 con 0x7f551c1079c0 2024-03-24T12:09:48.041 INFO:teuthology.orchestra.run.smithi090.stdout:[client.1] 2024-03-24T12:09:48.041 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQAMGABmseIzAhAA68SuJ2hIlUZnZBDUmTEsgg== 2024-03-24T12:09:48.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:48.044+0000 7f5522124700 1 -- 172.21.15.90:0/3788305781 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f550406d6b0 msgr2=0x7f550406fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:48.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:48.044+0000 7f5522124700 1 --2- 172.21.15.90:0/3788305781 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f550406d6b0 0x7f550406fb70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f550c00b5c0 tx=0x7f550c005fd0 comp rx=0 tx=0).stop 2024-03-24T12:09:48.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:48.044+0000 7f5522124700 1 -- 172.21.15.90:0/3788305781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f551c1079c0 msgr2=0x7f551c19e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:48.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:48.044+0000 7f5522124700 1 --2- 172.21.15.90:0/3788305781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f551c1079c0 0x7f551c19e720 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f551400b9e0 tx=0x7f551400bcf0 comp rx=0 tx=0).stop 2024-03-24T12:09:48.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:48.044+0000 7f5522124700 1 -- 172.21.15.90:0/3788305781 shutdown_connections 2024-03-24T12:09:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:48.044+0000 7f5522124700 1 --2- 172.21.15.90:0/3788305781 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f550406d6b0 0x7f550406fb70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:48.044+0000 7f5522124700 1 --2- 172.21.15.90:0/3788305781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f551c1079c0 0x7f551c19e720 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:48.044+0000 7f5522124700 1 --2- 172.21.15.90:0/3788305781 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f551c102760 0x7f551c1971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:48.044+0000 7f5522124700 1 --2- 172.21.15.90:0/3788305781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f551c105090 0x7f551c1976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:48.044+0000 7f5522124700 1 -- 172.21.15.90:0/3788305781 >> 172.21.15.90:0/3788305781 conn(0x7f551c0fc190 msgr2=0x7f551c1033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:48.044+0000 7f5522124700 1 -- 172.21.15.90:0/3788305781 shutdown_connections 2024-03-24T12:09:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-24T12:09:48.044+0000 7f5522124700 1 -- 172.21.15.90:0/3788305781 wait complete. 2024-03-24T12:09:48.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:47 smithi006 ceph-mon[30108]: osdmap e71: 8 total, 8 up, 8 in 2024-03-24T12:09:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:47 smithi006 ceph-mon[30108]: pgmap v212: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:09:48.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:47 smithi006 ceph-mon[41734]: osdmap e71: 8 total, 8 up, 8 in 2024-03-24T12:09:48.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:47 smithi006 ceph-mon[41734]: pgmap v212: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:09:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.90:0/3788305781' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T12:09:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:48 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T12:09:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:48 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-24T12:09:49.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.90:0/3788305781' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T12:09:49.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:48 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T12:09:49.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:48 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-24T12:09:49.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.90:0/3788305781' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T12:09:49.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:48 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-24T12:09:49.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:48 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-24T12:09:49.965 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-24T12:09:49.965 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-03-24T12:09:49.965 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-24T12:09:50.052 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-24T12:09:50.052 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-24T12:09:50.053 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph mgr dump --format=json 2024-03-24T12:09:50.200 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:49 smithi090 ceph-mon[33646]: pgmap v213: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:09:50.306 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:09:50.421 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:49 smithi006 ceph-mon[30108]: pgmap v213: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:09:50.422 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:49 smithi006 ceph-mon[41734]: pgmap v213: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:09:51.926 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:51 smithi090 systemd[1]: Starting Ceph prometheus.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:09:51.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.981+0000 7f5ab3257700 1 -- 172.21.15.6:0/1325964702 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5aac103930 msgr2=0x7f5aac103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:51.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.981+0000 7f5ab3257700 1 --2- 172.21.15.6:0/1325964702 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5aac103930 0x7f5aac103db0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009fc0 tx=0x7f5a9c00b330 comp rx=0 tx=0).stop 2024-03-24T12:09:51.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.981+0000 7f5ab3257700 1 -- 172.21.15.6:0/1325964702 shutdown_connections 2024-03-24T12:09:51.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.981+0000 7f5ab3257700 1 --2- 172.21.15.6:0/1325964702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5aac1042f0 0x7f5aac10eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:51.983 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.981+0000 7f5ab3257700 1 --2- 172.21.15.6:0/1325964702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5aac06a6c0 0x7f5aac06aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:51.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.981+0000 7f5ab3257700 1 --2- 172.21.15.6:0/1325964702 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5aac103930 0x7f5aac103db0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:51.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.981+0000 7f5ab3257700 1 -- 172.21.15.6:0/1325964702 >> 172.21.15.6:0/1325964702 conn(0x7f5aac076c10 msgr2=0x7f5aac077020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:51.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.981+0000 7f5ab3257700 1 -- 172.21.15.6:0/1325964702 shutdown_connections 2024-03-24T12:09:51.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.981+0000 7f5ab3257700 1 -- 172.21.15.6:0/1325964702 wait complete. 2024-03-24T12:09:51.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.982+0000 7f5ab3257700 1 Processor -- start 2024-03-24T12:09:51.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.982+0000 7f5ab3257700 1 -- start start 2024-03-24T12:09:51.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.982+0000 7f5ab3257700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5aac06a6c0 0x7f5aac19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:51.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.982+0000 7f5ab3257700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5aac103930 0x7f5aac19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:51.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.982+0000 7f5ab3257700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5aac1042f0 0x7f5aac197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:51.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.982+0000 7f5ab3257700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac197ee0 con 0x7f5aac1042f0 2024-03-24T12:09:51.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.982+0000 7f5ab3257700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac198020 con 0x7f5aac06a6c0 2024-03-24T12:09:51.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.982+0000 7f5ab3257700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac1981e0 con 0x7f5aac103930 2024-03-24T12:09:51.984 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.982+0000 7f5ab17f4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5aac1042f0 0x7f5aac197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:51.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.982+0000 7f5aabfff700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5aac103930 0x7f5aac19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:51.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.982+0000 7f5aabfff700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5aac103930 0x7f5aac19dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:47530/0 (socket says 172.21.15.6:47530) 2024-03-24T12:09:51.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.982+0000 7f5ab17f4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5aac1042f0 0x7f5aac197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37768/0 (socket says 172.21.15.6:37768) 2024-03-24T12:09:51.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.982+0000 7f5aabfff700 1 -- 172.21.15.6:0/3451296677 learned_addr learned my addr 172.21.15.6:0/3451296677 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:09:51.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.982+0000 7f5ab0ff3700 1 --2- 172.21.15.6:0/3451296677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5aac06a6c0 0x7f5aac19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:51.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.982+0000 7f5aabfff700 1 -- 172.21.15.6:0/3451296677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5aac06a6c0 msgr2=0x7f5aac19d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:51.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.982+0000 7f5aabfff700 1 --2- 172.21.15.6:0/3451296677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5aac06a6c0 0x7f5aac19d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:51.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.982+0000 7f5aabfff700 1 -- 172.21.15.6:0/3451296677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5aac1042f0 msgr2=0x7f5aac197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:51.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.982+0000 7f5aabfff700 1 --2- 172.21.15.6:0/3451296677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5aac1042f0 0x7f5aac197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:51.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.982+0000 7f5aabfff700 1 -- 172.21.15.6:0/3451296677 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a940095f0 con 0x7f5aac103930 2024-03-24T12:09:51.985 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.982+0000 7f5ab17f4700 1 --2- 172.21.15.6:0/3451296677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5aac1042f0 0x7f5aac197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:09:51.985 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:51 smithi006 ceph-mon[30108]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 41 B/s, 1 objects/s recovering 2024-03-24T12:09:51.986 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.983+0000 7f5aabfff700 1 --2- 172.21.15.6:0/3451296677 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5aac103930 0x7f5aac19dc00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c004fd0 tx=0x7f5a9c0079d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:51.986 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:51 smithi006 ceph-mon[41734]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 41 B/s, 1 objects/s recovering 2024-03-24T12:09:51.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.983+0000 7f5aa9ffb700 1 -- 172.21.15.6:0/3451296677 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c00e040 con 0x7f5aac103930 2024-03-24T12:09:51.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.983+0000 7f5aa9ffb700 1 -- 172.21.15.6:0/3451296677 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f5a9c007e40 con 0x7f5aac103930 2024-03-24T12:09:51.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.983+0000 7f5ab3257700 1 -- 172.21.15.6:0/3451296677 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9c009c40 con 0x7f5aac103930 2024-03-24T12:09:51.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.983+0000 7f5aa9ffb700 1 -- 172.21.15.6:0/3451296677 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c005040 con 0x7f5aac103930 2024-03-24T12:09:51.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.983+0000 7f5ab3257700 1 -- 172.21.15.6:0/3451296677 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aac1a4590 con 0x7f5aac103930 2024-03-24T12:09:51.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.984+0000 7f5ab3257700 1 -- 172.21.15.6:0/3451296677 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aac103440 con 0x7f5aac103930 2024-03-24T12:09:51.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.985+0000 7f5aa9ffb700 1 -- 172.21.15.6:0/3451296677 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 16) v1 ==== 91130+0+0 (secure 0 0 0) 0x7f5a9c0051a0 con 0x7f5aac103930 2024-03-24T12:09:51.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.985+0000 7f5aa9ffb700 1 --2- 172.21.15.6:0/3451296677 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f5a9806d650 0x7f5a9806fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:51.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.985+0000 7f5aa9ffb700 1 -- 172.21.15.6:0/3451296677 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7207+0+0 (secure 0 0 0) 0x7f5a9c08e530 con 0x7f5aac103930 2024-03-24T12:09:51.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.985+0000 7f5ab0ff3700 1 --2- 172.21.15.6:0/3451296677 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f5a9806d650 0x7f5a9806fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:51.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.986+0000 7f5ab0ff3700 1 --2- 172.21.15.6:0/3451296677 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f5a9806d650 0x7f5a9806fb10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5aac1989c0 tx=0x7f5a94009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:51.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:51.990+0000 7f5aa9ffb700 1 -- 172.21.15.6:0/3451296677 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5a9c057cb0 con 0x7f5aac103930 2024-03-24T12:09:52.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:51 smithi090 ceph-mon[33646]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 41 B/s, 1 objects/s recovering 2024-03-24T12:09:52.178 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:51 smithi090 podman[64668]: 2024-03-24 12:09:51.830742022 +0000 UTC m=+0.018993419 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-03-24T12:09:52.178 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:51 smithi090 podman[64668]: 2024-03-24 12:09:51.947062579 +0000 UTC m=+0.135313962 container create 3bb67e0745abf3e70a7aca2c44ea0b0e3f493aee962755df2a0db4cc2e91918b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T12:09:52.178 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 podman[64668]: 2024-03-24 12:09:52.168345789 +0000 UTC m=+0.356597182 container init 3bb67e0745abf3e70a7aca2c44ea0b0e3f493aee962755df2a0db4cc2e91918b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T12:09:52.179 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 podman[64668]: 2024-03-24 12:09:52.173776851 +0000 UTC m=+0.362028237 container start 3bb67e0745abf3e70a7aca2c44ea0b0e3f493aee962755df2a0db4cc2e91918b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T12:09:52.253 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:52.252+0000 7f5ab3257700 1 -- 172.21.15.6:0/3451296677 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f5aac198db0 con 0x7f5aac103930 2024-03-24T12:09:52.256 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:52.256+0000 7f5aa9ffb700 1 -- 172.21.15.6:0/3451296677 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v16) v1 ==== 74+0+174300 (secure 0 0 0) 0x7f5a9c00be60 con 0x7f5aac103930 2024-03-24T12:09:52.259 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:09:52.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:52.258+0000 7f5ab3257700 1 -- 172.21.15.6:0/3451296677 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f5a9806d650 msgr2=0x7f5a9806fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:52.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:52.258+0000 7f5ab3257700 1 --2- 172.21.15.6:0/3451296677 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f5a9806d650 0x7f5a9806fb10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5aac1989c0 tx=0x7f5a94009310 comp rx=0 tx=0).stop 2024-03-24T12:09:52.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:52.258+0000 7f5ab3257700 1 -- 172.21.15.6:0/3451296677 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5aac103930 msgr2=0x7f5aac19dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:52.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:52.258+0000 7f5ab3257700 1 --2- 172.21.15.6:0/3451296677 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5aac103930 0x7f5aac19dc00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c004fd0 tx=0x7f5a9c0079d0 comp rx=0 tx=0).stop 2024-03-24T12:09:52.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:52.259+0000 7f5ab3257700 1 -- 172.21.15.6:0/3451296677 shutdown_connections 2024-03-24T12:09:52.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:52.259+0000 7f5ab3257700 1 --2- 172.21.15.6:0/3451296677 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f5a9806d650 0x7f5a9806fb10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:52.260 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:52.259+0000 7f5ab3257700 1 --2- 172.21.15.6:0/3451296677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5aac1042f0 0x7f5aac197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:52.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:52.259+0000 7f5ab3257700 1 --2- 172.21.15.6:0/3451296677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5aac06a6c0 0x7f5aac19d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:52.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:52.259+0000 7f5ab3257700 1 --2- 172.21.15.6:0/3451296677 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5aac103930 0x7f5aac19dc00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:52.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:52.259+0000 7f5ab3257700 1 -- 172.21.15.6:0/3451296677 >> 172.21.15.6:0/3451296677 conn(0x7f5aac076c10 msgr2=0x7f5aac106a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:52.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:52.259+0000 7f5ab3257700 1 -- 172.21.15.6:0/3451296677 shutdown_connections 2024-03-24T12:09:52.261 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:52.259+0000 7f5ab3257700 1 -- 172.21.15.6:0/3451296677 wait complete. 2024-03-24T12:09:52.446 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:09:52.196Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-03-24T12:09:52.447 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:09:52.196Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-03-24T12:09:52.447 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:09:52.197Z caller=main.go:570 level=info host_details="(Linux 5.14.0-430.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024 x86_64 smithi090 (none))" 2024-03-24T12:09:52.447 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:09:52.197Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-03-24T12:09:52.447 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:09:52.197Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-03-24T12:09:52.447 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:09:52.198Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-03-24T12:09:52.447 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:09:52.198Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-03-24T12:09:52.447 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:09:52.199Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-03-24T12:09:52.447 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:09:52.199Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-03-24T12:09:52.447 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:09:52.201Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-03-24T12:09:52.447 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:09:52.201Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.539µs 2024-03-24T12:09:52.447 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:09:52.201Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-03-24T12:09:52.447 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:09:52.201Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-03-24T12:09:52.447 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:09:52.201Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=16.657µs wal_replay_duration=210.648µs wbl_replay_duration=121ns total_replay_duration=245.06µs 2024-03-24T12:09:52.447 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:09:52.202Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-03-24T12:09:52.448 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:09:52.202Z caller=main.go:1029 level=info msg="TSDB started" 2024-03-24T12:09:52.448 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:09:52.202Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-03-24T12:09:52.448 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:09:52.220Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=17.85619ms db_storage=735ns remote_storage=1.011µs web_handler=346ns query_engine=607ns scrape=471.654µs scrape_sd=72.984µs notify=709ns notify_sd=1.197µs rules=17.136907ms tracing=6.407µs 2024-03-24T12:09:52.448 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:09:52.220Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-03-24T12:09:52.448 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:09:52.220Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-03-24T12:09:52.448 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 bash[64668]: 3bb67e0745abf3e70a7aca2c44ea0b0e3f493aee962755df2a0db4cc2e91918b 2024-03-24T12:09:52.928 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:09:52 smithi090 systemd[1]: Started Ceph prometheus.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:09:52.982 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":16,"flags":0,"active_gid":14152,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3847865179},{"type":"v1","addr":"172.21.15.6:6801","nonce":3847865179}]},"active_addr":"172.21.15.6:6801/3847865179","active_change":"2024-03-24T12:04:45.837010+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.6:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":1484838563}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":38969824}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":2556976022}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":369643444}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":3307470923}]}]} 2024-03-24T12:09:52.986 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-24T12:09:52.986 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-24T12:09:52.986 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-24T12:09:53.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:52 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3451296677' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-24T12:09:53.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:52 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:53.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:52 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:53.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:52 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:53.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:52 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-24T12:09:53.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:52 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3451296677' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-24T12:09:53.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:52 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:53.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:52 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:53.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:52 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:53.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:52 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-24T12:09:53.230 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:09:53.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3451296677' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-24T12:09:53.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:53.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:53.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' 2024-03-24T12:09:53.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:52 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-24T12:09:53.664 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:09:53 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ignoring --setuser ceph since I am not root 2024-03-24T12:09:53.664 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:09:53 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ignoring --setgroup ceph since I am not root 2024-03-24T12:09:53.928 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:53 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: ignoring --setuser ceph since I am not root 2024-03-24T12:09:53.928 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:53 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: ignoring --setgroup ceph since I am not root 2024-03-24T12:09:53.928 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:53 smithi090 ceph-mgr[35739]: -- 172.21.15.90:0/1048882530 <== mon.1 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cc69f261a0 con 0x55cc69efd000 2024-03-24T12:09:54.393 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:09:54 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:09:54.107+0000 7f94c23d1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T12:09:54.393 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:09:54 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:09:54.203+0000 7f94c23d1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T12:09:54.428 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:54 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:09:54.085+0000 7f05cbb92200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-24T12:09:54.428 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:54 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:09:54.179+0000 7f05cbb92200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-24T12:09:54.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:54 smithi006 ceph-mon[30108]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-24T12:09:54.672 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:54 smithi006 ceph-mon[30108]: mgrmap e17: y(active, since 5m), standbys: x 2024-03-24T12:09:54.673 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:54 smithi006 ceph-mon[41734]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-24T12:09:54.673 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:54 smithi006 ceph-mon[41734]: mgrmap e17: y(active, since 5m), standbys: x 2024-03-24T12:09:54.811 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:54 smithi090 ceph-mon[33646]: from='mgr.14152 172.21.15.6:0/3705755563' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-24T12:09:54.812 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:54 smithi090 ceph-mon[33646]: mgrmap e17: y(active, since 5m), standbys: x 2024-03-24T12:09:54.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.929+0000 7f6dd90c9700 1 -- 172.21.15.6:0/3968315393 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6dd41015d0 msgr2=0x7f6dd41019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:54.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.929+0000 7f6dd90c9700 1 --2- 172.21.15.6:0/3968315393 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6dd41015d0 0x7f6dd41019b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009990 tx=0x7f6dc4009ca0 comp rx=0 tx=0).stop 2024-03-24T12:09:54.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.929+0000 7f6dd90c9700 1 -- 172.21.15.6:0/3968315393 shutdown_connections 2024-03-24T12:09:54.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.929+0000 7f6dd90c9700 1 --2- 172.21.15.6:0/3968315393 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6dd4105ef0 0x7f6dd410dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:54.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.929+0000 7f6dd90c9700 1 --2- 172.21.15.6:0/3968315393 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6dd41015d0 0x7f6dd41019b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:54.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.929+0000 7f6dd90c9700 1 --2- 172.21.15.6:0/3968315393 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6dd4101f80 0x7f6dd4105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:54.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.929+0000 7f6dd90c9700 1 -- 172.21.15.6:0/3968315393 >> 172.21.15.6:0/3968315393 conn(0x7f6dd4077170 msgr2=0x7f6dd40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:54.929 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.929+0000 7f6dd90c9700 1 -- 172.21.15.6:0/3968315393 shutdown_connections 2024-03-24T12:09:54.929 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:09:54 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:09:54.875+0000 7f94c23d1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T12:09:54.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.930+0000 7f6dd90c9700 1 -- 172.21.15.6:0/3968315393 wait complete. 2024-03-24T12:09:54.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.930+0000 7f6dd90c9700 1 Processor -- start 2024-03-24T12:09:54.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.931+0000 7f6dd90c9700 1 -- start start 2024-03-24T12:09:54.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.931+0000 7f6dd90c9700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6dd41015d0 0x7f6dd40fc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:54.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.931+0000 7f6dd90c9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6dd4101f80 0x7f6dd40fcb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:54.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.931+0000 7f6dd90c9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6dd4105ef0 0x7f6dd40fd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:54.931 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.931+0000 7f6dd90c9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd41a5b90 con 0x7f6dd4105ef0 2024-03-24T12:09:54.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.931+0000 7f6dd90c9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd41a5d00 con 0x7f6dd4101f80 2024-03-24T12:09:54.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.931+0000 7f6dd90c9700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd41a5ec0 con 0x7f6dd41015d0 2024-03-24T12:09:54.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.931+0000 7f6dd2d9d700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6dd41015d0 0x7f6dd40fc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:54.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.931+0000 7f6dd359e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6dd4105ef0 0x7f6dd40fd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:54.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.931+0000 7f6dd2d9d700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6dd41015d0 0x7f6dd40fc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:47560/0 (socket says 172.21.15.6:47560) 2024-03-24T12:09:54.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.931+0000 7f6dd2d9d700 1 -- 172.21.15.6:0/3684200946 learned_addr learned my addr 172.21.15.6:0/3684200946 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:09:54.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.931+0000 7f6dd259c700 1 --2- 172.21.15.6:0/3684200946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6dd4101f80 0x7f6dd40fcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:54.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.931+0000 7f6dd2d9d700 1 -- 172.21.15.6:0/3684200946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6dd4101f80 msgr2=0x7f6dd40fcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:54.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.931+0000 7f6dd2d9d700 1 --2- 172.21.15.6:0/3684200946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6dd4101f80 0x7f6dd40fcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:54.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.931+0000 7f6dd2d9d700 1 -- 172.21.15.6:0/3684200946 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6dd4105ef0 msgr2=0x7f6dd40fd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:54.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.931+0000 7f6dd2d9d700 1 --2- 172.21.15.6:0/3684200946 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6dd4105ef0 0x7f6dd40fd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:54.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.931+0000 7f6dd2d9d700 1 -- 172.21.15.6:0/3684200946 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc009c40 con 0x7f6dd41015d0 2024-03-24T12:09:54.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.931+0000 7f6dd259c700 1 --2- 172.21.15.6:0/3684200946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6dd4101f80 0x7f6dd40fcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:09:54.932 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.931+0000 7f6dd359e700 1 --2- 172.21.15.6:0/3684200946 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6dd4105ef0 0x7f6dd40fd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:09:54.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.932+0000 7f6dd2d9d700 1 --2- 172.21.15.6:0/3684200946 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6dd41015d0 0x7f6dd40fc650 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4004760 tx=0x7f6dc4004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:54.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.932+0000 7f6dcbfff700 1 -- 172.21.15.6:0/3684200946 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc401d070 con 0x7f6dd41015d0 2024-03-24T12:09:54.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.932+0000 7f6dd90c9700 1 -- 172.21.15.6:0/3684200946 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc40095f0 con 0x7f6dd41015d0 2024-03-24T12:09:54.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.932+0000 7f6dcbfff700 1 -- 172.21.15.6:0/3684200946 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f6dc4005a00 con 0x7f6dd41015d0 2024-03-24T12:09:54.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.932+0000 7f6dcbfff700 1 -- 172.21.15.6:0/3684200946 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4005020 con 0x7f6dd41015d0 2024-03-24T12:09:54.933 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.932+0000 7f6dd90c9700 1 -- 172.21.15.6:0/3684200946 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd41a63f0 con 0x7f6dd41015d0 2024-03-24T12:09:54.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.933+0000 7f6dd90c9700 1 -- 172.21.15.6:0/3684200946 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd410a120 con 0x7f6dd41015d0 2024-03-24T12:09:54.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.933+0000 7f6dcbfff700 1 -- 172.21.15.6:0/3684200946 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 17) v1 ==== 91144+0+0 (secure 0 0 0) 0x7f6dc400b920 con 0x7f6dd41015d0 2024-03-24T12:09:54.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.934+0000 7f6dcbfff700 1 --2- 172.21.15.6:0/3684200946 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f6dc006d650 0x7f6dc006fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:54.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.934+0000 7f6dd259c700 1 -- 172.21.15.6:0/3684200946 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f6dc006d650 msgr2=0x7f6dc006fb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3847865179 2024-03-24T12:09:54.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.934+0000 7f6dcbfff700 1 -- 172.21.15.6:0/3684200946 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7207+0+0 (secure 0 0 0) 0x7f6dc408e140 con 0x7f6dd41015d0 2024-03-24T12:09:54.934 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.934+0000 7f6dd259c700 1 --2- 172.21.15.6:0/3684200946 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f6dc006d650 0x7f6dc006fb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T12:09:54.938 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:54.938+0000 7f6dcbfff700 1 -- 172.21.15.6:0/3684200946 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6dc40577c0 con 0x7f6dd41015d0 2024-03-24T12:09:55.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:55.135+0000 7f6dd259c700 1 -- 172.21.15.6:0/3684200946 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f6dc006d650 msgr2=0x7f6dc006fb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3847865179 2024-03-24T12:09:55.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:55.135+0000 7f6dd259c700 1 --2- 172.21.15.6:0/3684200946 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f6dc006d650 0x7f6dc006fb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-24T12:09:55.146 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:54 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:09:54.812+0000 7f05cbb92200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-24T12:09:55.147 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:54 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:09:54.904+0000 7f05cbb92200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T12:09:55.147 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:55 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:09:55.048+0000 7f05cbb92200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T12:09:55.147 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:55 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:09:55.147+0000 7f05cbb92200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T12:09:55.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:55.172+0000 7f6dd90c9700 1 -- 172.21.15.6:0/3684200946 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6dd40fdb80 con 0x7f6dd41015d0 2024-03-24T12:09:55.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:55.174+0000 7f6dcbfff700 1 -- 172.21.15.6:0/3684200946 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+22623 (secure 0 0 0) 0x7f6dc405ae80 con 0x7f6dd41015d0 2024-03-24T12:09:55.173 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:09:55.173 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":71,"fsid":"66d0ad74-e9d6-11ee-95cd-87774f69a715","created":"2024-03-24T12:03:42.337185+0000","modified":"2024-03-24T12:09:46.947032+0000","last_up_change":"2024-03-24T12:09:06.451078+0000","last_in_change":"2024-03-24T12:08:48.739062+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T12:07:02.169336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-03-24T12:09:18.673658+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":64,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-03-24T12:09:19.237707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-03-24T12:09:21.759379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-03-24T12:09:23.341532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-03-24T12:09:25.415111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3542a64b-e3c1-4a04-ba1c-bdfbcd4a0aa4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":195441355},{"type":"v1","addr":"172.21.15.6:6803","nonce":195441355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":195441355},{"type":"v1","addr":"172.21.15.6:6805","nonce":195441355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":195441355},{"type":"v1","addr":"172.21.15.6:6809","nonce":195441355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":195441355},{"type":"v1","addr":"172.21.15.6:6807","nonce":195441355}]},"public_addr":"172.21.15.6:6803/195441355","cluster_addr":"172.21.15.6:6805/195441355","heartbeat_back_addr":"172.21.15.6:6809/195441355","heartbeat_front_addr":"172.21.15.6:6807/195441355","state":["exists","up"]},{"osd":1,"uuid":"b00997af-0f11-4f04-ae58-81af568256cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3140266601},{"type":"v1","addr":"172.21.15.6:6811","nonce":3140266601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3140266601},{"type":"v1","addr":"172.21.15.6:6813","nonce":3140266601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3140266601},{"type":"v1","addr":"172.21.15.6:6817","nonce":3140266601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3140266601},{"type":"v1","addr":"172.21.15.6:6815","nonce":3140266601}]},"public_addr":"172.21.15.6:6811/3140266601","cluster_addr":"172.21.15.6:6813/3140266601","heartbeat_back_addr":"172.21.15.6:6817/3140266601","heartbeat_front_addr":"172.21.15.6:6815/3140266601","state":["exists","up"]},{"osd":2,"uuid":"eec34136-82f9-41eb-bbac-f0d6edf7e73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":4008388154},{"type":"v1","addr":"172.21.15.6:6819","nonce":4008388154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":4008388154},{"type":"v1","addr":"172.21.15.6:6821","nonce":4008388154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":4008388154},{"type":"v1","addr":"172.21.15.6:6825","nonce":4008388154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":4008388154},{"type":"v1","addr":"172.21.15.6:6823","nonce":4008388154}]},"public_addr":"172.21.15.6:6819/4008388154","cluster_addr":"172.21.15.6:6821/4008388154","heartbeat_back_addr":"172.21.15.6:6825/4008388154","heartbeat_front_addr":"172.21.15.6:6823/4008388154","state":["exists","up"]},{"osd":3,"uuid":"bd1c26cc-c121-4570-87ff-f432ecc53e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1341863884},{"type":"v1","addr":"172.21.15.6:6827","nonce":1341863884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1341863884},{"type":"v1","addr":"172.21.15.6:6829","nonce":1341863884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1341863884},{"type":"v1","addr":"172.21.15.6:6833","nonce":1341863884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1341863884},{"type":"v1","addr":"172.21.15.6:6831","nonce":1341863884}]},"public_addr":"172.21.15.6:6827/1341863884","cluster_addr":"172.21.15.6:6829/1341863884","heartbeat_back_addr":"172.21.15.6:6833/1341863884","heartbeat_front_addr":"172.21.15.6:6831/1341863884","state":["exists","up"]},{"osd":4,"uuid":"65a98356-0941-41e8-a10b-9d0f76212fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3147977284},{"type":"v1","addr":"172.21.15.90:6801","nonce":3147977284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3147977284},{"type":"v1","addr":"172.21.15.90:6803","nonce":3147977284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3147977284},{"type":"v1","addr":"172.21.15.90:6807","nonce":3147977284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3147977284},{"type":"v1","addr":"172.21.15.90:6805","nonce":3147977284}]},"public_addr":"172.21.15.90:6801/3147977284","cluster_addr":"172.21.15.90:6803/3147977284","heartbeat_back_addr":"172.21.15.90:6807/3147977284","heartbeat_front_addr":"172.21.15.90:6805/3147977284","state":["exists","up"]},{"osd":5,"uuid":"b143b33c-9447-4b8c-9991-5037f50789ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":3585923902},{"type":"v1","addr":"172.21.15.90:6809","nonce":3585923902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3585923902},{"type":"v1","addr":"172.21.15.90:6811","nonce":3585923902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3585923902},{"type":"v1","addr":"172.21.15.90:6815","nonce":3585923902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3585923902},{"type":"v1","addr":"172.21.15.90:6813","nonce":3585923902}]},"public_addr":"172.21.15.90:6809/3585923902","cluster_addr":"172.21.15.90:6811/3585923902","heartbeat_back_addr":"172.21.15.90:6815/3585923902","heartbeat_front_addr":"172.21.15.90:6813/3585923902","state":["exists","up"]},{"osd":6,"uuid":"43226b7c-1700-47e4-8d1b-7f765e8defda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":713247979},{"type":"v1","addr":"172.21.15.90:6817","nonce":713247979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":713247979},{"type":"v1","addr":"172.21.15.90:6819","nonce":713247979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":713247979},{"type":"v1","addr":"172.21.15.90:6823","nonce":713247979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":713247979},{"type":"v1","addr":"172.21.15.90:6821","nonce":713247979}]},"public_addr":"172.21.15.90:6817/713247979","cluster_addr":"172.21.15.90:6819/713247979","heartbeat_back_addr":"172.21.15.90:6823/713247979","heartbeat_front_addr":"172.21.15.90:6821/713247979","state":["exists","up"]},{"osd":7,"uuid":"fa0def8c-eda9-441c-8ee9-e0021c064bf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1629523727},{"type":"v1","addr":"172.21.15.90:6825","nonce":1629523727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1629523727},{"type":"v1","addr":"172.21.15.90:6827","nonce":1629523727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1629523727},{"type":"v1","addr":"172.21.15.90:6831","nonce":1629523727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1629523727},{"type":"v1","addr":"172.21.15.90:6829","nonce":1629523727}]},"public_addr":"172.21.15.90:6825/1629523727","cluster_addr":"172.21.15.90:6827/1629523727","heartbeat_back_addr":"172.21.15.90:6831/1629523727","heartbeat_front_addr":"172.21.15.90:6829/1629523727","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:06:08.730123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:06:33.621931+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:06:58.862101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:07:25.210912+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:07:49.154431+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:08:14.240635+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:08:39.558437+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:09:04.997129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1159789551":"2024-03-25T12:04:45.836851+0000","172.21.15.6:0/2683803593":"2024-03-25T12:04:45.836851+0000","172.21.15.6:6801/3010671792":"2024-03-25T12:04:45.836851+0000","172.21.15.6:6800/3010671792":"2024-03-25T12:04:45.836851+0000","172.21.15.6:6801/2054386689":"2024-03-25T12:04:18.792992+0000","172.21.15.6:6800/2054386689":"2024-03-25T12:04:18.792992+0000","172.21.15.6:0/897967172":"2024-03-25T12:04:18.792992+0000","172.21.15.6:0/2335021214":"2024-03-25T12:04:18.792992+0000","172.21.15.6:0/3268002752":"2024-03-25T12:04:18.792992+0000","172.21.15.6:0/1607559555":"2024-03-25T12:04:45.836851+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T12:09:55.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:55.176+0000 7f6dd90c9700 1 -- 172.21.15.6:0/3684200946 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f6dc006d650 msgr2=0x7f6dc006fb10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T12:09:55.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:55.176+0000 7f6dd90c9700 1 --2- 172.21.15.6:0/3684200946 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f6dc006d650 0x7f6dc006fb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:55.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:55.176+0000 7f6dd90c9700 1 -- 172.21.15.6:0/3684200946 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6dd41015d0 msgr2=0x7f6dd40fc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:55.175 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:55.176+0000 7f6dd90c9700 1 --2- 172.21.15.6:0/3684200946 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6dd41015d0 0x7f6dd40fc650 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4004760 tx=0x7f6dc4004790 comp rx=0 tx=0).stop 2024-03-24T12:09:55.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:55.176+0000 7f6dd90c9700 1 -- 172.21.15.6:0/3684200946 shutdown_connections 2024-03-24T12:09:55.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:55.176+0000 7f6dd90c9700 1 --2- 172.21.15.6:0/3684200946 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7f6dc006d650 0x7f6dc006fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:55.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:55.176+0000 7f6dd90c9700 1 --2- 172.21.15.6:0/3684200946 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6dd4105ef0 0x7f6dd40fd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:55.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:55.176+0000 7f6dd90c9700 1 --2- 172.21.15.6:0/3684200946 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6dd41015d0 0x7f6dd40fc650 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:55.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:55.176+0000 7f6dd90c9700 1 --2- 172.21.15.6:0/3684200946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6dd4101f80 0x7f6dd40fcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:55.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:55.176+0000 7f6dd90c9700 1 -- 172.21.15.6:0/3684200946 >> 172.21.15.6:0/3684200946 conn(0x7f6dd4077170 msgr2=0x7f6dd40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:55.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:55.176+0000 7f6dd90c9700 1 -- 172.21.15.6:0/3684200946 shutdown_connections 2024-03-24T12:09:55.176 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:55.176+0000 7f6dd90c9700 1 -- 172.21.15.6:0/3684200946 wait complete. 2024-03-24T12:09:55.187 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:09:54 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:09:54.997+0000 7f94c23d1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-24T12:09:55.187 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:09:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:09:55.143+0000 7f94c23d1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-24T12:09:55.428 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:55 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:09:55.230+0000 7f05cbb92200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T12:09:55.460 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:09:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:09:55.248+0000 7f94c23d1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-24T12:09:55.461 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:09:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:09:55.333+0000 7f94c23d1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-24T12:09:55.786 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:55 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3684200946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T12:09:55.787 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:55 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:09:55.496+0000 7f05cbb92200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-24T12:09:55.787 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:55 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:09:55.608+0000 7f05cbb92200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T12:09:55.787 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:55 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:09:55.693+0000 7f05cbb92200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T12:09:55.787 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:55 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:09:55.787+0000 7f05cbb92200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T12:09:55.809 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:55 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3684200946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T12:09:55.809 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:55 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3684200946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T12:09:55.809 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:09:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:09:55.608+0000 7f94c23d1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-24T12:09:55.810 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:09:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:09:55.721+0000 7f94c23d1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-24T12:09:55.862 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-24T12:09:55.862 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-24T12:09:56.108 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:09:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:09:55.810+0000 7f94c23d1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-24T12:09:56.108 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:09:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:09:55.911+0000 7f94c23d1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-24T12:09:56.108 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:09:56 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:09:56.108+0000 7f94c23d1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T12:09:56.128 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:09:56.178 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:55 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:09:55.962+0000 7f05cbb92200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-24T12:09:57.178 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:56 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:09:56.920+0000 7f05cbb92200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T12:09:57.178 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:57 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:09:57.049+0000 7f05cbb92200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T12:09:57.384 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:09:57 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:09:57.127+0000 7f94c23d1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-24T12:09:57.384 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:09:57 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:09:57.262+0000 7f94c23d1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-24T12:09:57.449 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:57 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:09:57.392+0000 7f05cbb92200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T12:09:57.635 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:09:57 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:09:57.610+0000 7f94c23d1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-24T12:09:57.737 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.736+0000 7ff5706b2700 1 -- 172.21.15.6:0/2408863065 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff568100ba0 msgr2=0x7ff568101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:57.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.736+0000 7ff5706b2700 1 --2- 172.21.15.6:0/2408863065 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff568100ba0 0x7ff568101020 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff5640098f0 tx=0x7ff564009c00 comp rx=0 tx=0).stop 2024-03-24T12:09:57.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.736+0000 7ff5706b2700 1 -- 172.21.15.6:0/2408863065 shutdown_connections 2024-03-24T12:09:57.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.736+0000 7ff5706b2700 1 --2- 172.21.15.6:0/2408863065 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff568101560 0x7ff56810d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:57.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.736+0000 7ff5706b2700 1 --2- 172.21.15.6:0/2408863065 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff568106bd0 0x7ff568106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:57.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.736+0000 7ff5706b2700 1 --2- 172.21.15.6:0/2408863065 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff568100ba0 0x7ff568101020 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:57.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.736+0000 7ff5706b2700 1 -- 172.21.15.6:0/2408863065 >> 172.21.15.6:0/2408863065 conn(0x7ff568076bc0 msgr2=0x7ff568076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:57.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.736+0000 7ff5706b2700 1 -- 172.21.15.6:0/2408863065 shutdown_connections 2024-03-24T12:09:57.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.736+0000 7ff5706b2700 1 -- 172.21.15.6:0/2408863065 wait complete. 2024-03-24T12:09:57.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.737+0000 7ff5706b2700 1 Processor -- start 2024-03-24T12:09:57.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.737+0000 7ff5706b2700 1 -- start start 2024-03-24T12:09:57.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.737+0000 7ff5706b2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff568100ba0 0x7ff568102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:57.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.737+0000 7ff5706b2700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff568101560 0x7ff568102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:57.738 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.737+0000 7ff5706b2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff568106bd0 0x7ff568103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:57.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.737+0000 7ff5706b2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5681a5b30 con 0x7ff568100ba0 2024-03-24T12:09:57.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.737+0000 7ff5706b2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5681a5ca0 con 0x7ff568106bd0 2024-03-24T12:09:57.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.737+0000 7ff5706b2700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5681a5e60 con 0x7ff568101560 2024-03-24T12:09:57.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.737+0000 7ff56dc4d700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff568101560 0x7ff568102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:57.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.737+0000 7ff56dc4d700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff568101560 0x7ff568102cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:59916/0 (socket says 172.21.15.6:59916) 2024-03-24T12:09:57.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.737+0000 7ff56dc4d700 1 -- 172.21.15.6:0/3955299007 learned_addr learned my addr 172.21.15.6:0/3955299007 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:09:57.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.737+0000 7ff56e44e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff568100ba0 0x7ff568102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:57.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.737+0000 7ff56ec4f700 1 --2- 172.21.15.6:0/3955299007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff568106bd0 0x7ff568103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:09:57.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.738+0000 7ff56e44e700 1 -- 172.21.15.6:0/3955299007 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff568101560 msgr2=0x7ff568102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:57.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.738+0000 7ff56e44e700 1 --2- 172.21.15.6:0/3955299007 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff568101560 0x7ff568102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:57.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.738+0000 7ff56e44e700 1 -- 172.21.15.6:0/3955299007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff568106bd0 msgr2=0x7ff568103210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:57.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.738+0000 7ff56e44e700 1 --2- 172.21.15.6:0/3955299007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff568106bd0 0x7ff568103210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:57.739 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.738+0000 7ff56e44e700 1 -- 172.21.15.6:0/3955299007 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5580095f0 con 0x7ff568100ba0 2024-03-24T12:09:57.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.738+0000 7ff56dc4d700 1 --2- 172.21.15.6:0/3955299007 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff568101560 0x7ff568102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:09:57.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.738+0000 7ff56e44e700 1 --2- 172.21.15.6:0/3955299007 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff568100ba0 0x7ff568102790 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff55800eb90 tx=0x7ff55800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:09:57.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.738+0000 7ff55f7fe700 1 -- 172.21.15.6:0/3955299007 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558009970 con 0x7ff568100ba0 2024-03-24T12:09:57.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.738+0000 7ff5706b2700 1 -- 172.21.15.6:0/3955299007 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff564009520 con 0x7ff568100ba0 2024-03-24T12:09:57.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.738+0000 7ff55f7fe700 1 -- 172.21.15.6:0/3955299007 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ff5580044a0 con 0x7ff568100ba0 2024-03-24T12:09:57.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.738+0000 7ff55f7fe700 1 -- 172.21.15.6:0/3955299007 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558003d40 con 0x7ff568100ba0 2024-03-24T12:09:57.740 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.738+0000 7ff5706b2700 1 -- 172.21.15.6:0/3955299007 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5681a6500 con 0x7ff568100ba0 2024-03-24T12:09:57.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.739+0000 7ff5706b2700 1 -- 172.21.15.6:0/3955299007 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff568068930 con 0x7ff568100ba0 2024-03-24T12:09:57.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.740+0000 7ff55f7fe700 1 -- 172.21.15.6:0/3955299007 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 17) v1 ==== 91144+0+0 (secure 0 0 0) 0x7ff558003ea0 con 0x7ff568100ba0 2024-03-24T12:09:57.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.740+0000 7ff55f7fe700 1 --2- 172.21.15.6:0/3955299007 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ff55406d650 0x7ff55406fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:09:57.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.741+0000 7ff56dc4d700 1 -- 172.21.15.6:0/3955299007 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ff55406d650 msgr2=0x7ff55406fb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3847865179 2024-03-24T12:09:57.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.741+0000 7ff55f7fe700 1 -- 172.21.15.6:0/3955299007 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7207+0+0 (secure 0 0 0) 0x7ff558014070 con 0x7ff568100ba0 2024-03-24T12:09:57.741 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.741+0000 7ff56dc4d700 1 --2- 172.21.15.6:0/3955299007 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ff55406d650 0x7ff55406fb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-24T12:09:57.749 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.750+0000 7ff55f7fe700 1 -- 172.21.15.6:0/3955299007 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff558056c80 con 0x7ff568100ba0 2024-03-24T12:09:57.893 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:57 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:09:57.489+0000 7f05cbb92200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T12:09:57.893 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:57 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:09:57.698+0000 7f05cbb92200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T12:09:57.918 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:09:57 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:09:57.707+0000 7f94c23d1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-24T12:09:57.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.942+0000 7ff56dc4d700 1 -- 172.21.15.6:0/3955299007 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ff55406d650 msgr2=0x7ff55406fb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.6:6800/3847865179 2024-03-24T12:09:57.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.942+0000 7ff56dc4d700 1 --2- 172.21.15.6:0/3955299007 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ff55406d650 0x7ff55406fb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-24T12:09:57.954 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.955+0000 7ff5706b2700 1 -- 172.21.15.6:0/3955299007 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5681100a0 con 0x7ff568100ba0 2024-03-24T12:09:57.957 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.958+0000 7ff55f7fe700 1 -- 172.21.15.6:0/3955299007 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+22623 (secure 0 0 0) 0x7ff55805a340 con 0x7ff568100ba0 2024-03-24T12:09:57.957 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:09:57.957 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":71,"fsid":"66d0ad74-e9d6-11ee-95cd-87774f69a715","created":"2024-03-24T12:03:42.337185+0000","modified":"2024-03-24T12:09:46.947032+0000","last_up_change":"2024-03-24T12:09:06.451078+0000","last_in_change":"2024-03-24T12:08:48.739062+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-24T12:07:02.169336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-03-24T12:09:18.673658+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":64,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-03-24T12:09:19.237707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-03-24T12:09:21.759379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-03-24T12:09:23.341532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-03-24T12:09:25.415111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3542a64b-e3c1-4a04-ba1c-bdfbcd4a0aa4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":195441355},{"type":"v1","addr":"172.21.15.6:6803","nonce":195441355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":195441355},{"type":"v1","addr":"172.21.15.6:6805","nonce":195441355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":195441355},{"type":"v1","addr":"172.21.15.6:6809","nonce":195441355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":195441355},{"type":"v1","addr":"172.21.15.6:6807","nonce":195441355}]},"public_addr":"172.21.15.6:6803/195441355","cluster_addr":"172.21.15.6:6805/195441355","heartbeat_back_addr":"172.21.15.6:6809/195441355","heartbeat_front_addr":"172.21.15.6:6807/195441355","state":["exists","up"]},{"osd":1,"uuid":"b00997af-0f11-4f04-ae58-81af568256cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":3140266601},{"type":"v1","addr":"172.21.15.6:6811","nonce":3140266601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":3140266601},{"type":"v1","addr":"172.21.15.6:6813","nonce":3140266601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6816","nonce":3140266601},{"type":"v1","addr":"172.21.15.6:6817","nonce":3140266601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":3140266601},{"type":"v1","addr":"172.21.15.6:6815","nonce":3140266601}]},"public_addr":"172.21.15.6:6811/3140266601","cluster_addr":"172.21.15.6:6813/3140266601","heartbeat_back_addr":"172.21.15.6:6817/3140266601","heartbeat_front_addr":"172.21.15.6:6815/3140266601","state":["exists","up"]},{"osd":2,"uuid":"eec34136-82f9-41eb-bbac-f0d6edf7e73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":4008388154},{"type":"v1","addr":"172.21.15.6:6819","nonce":4008388154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":4008388154},{"type":"v1","addr":"172.21.15.6:6821","nonce":4008388154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":4008388154},{"type":"v1","addr":"172.21.15.6:6825","nonce":4008388154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":4008388154},{"type":"v1","addr":"172.21.15.6:6823","nonce":4008388154}]},"public_addr":"172.21.15.6:6819/4008388154","cluster_addr":"172.21.15.6:6821/4008388154","heartbeat_back_addr":"172.21.15.6:6825/4008388154","heartbeat_front_addr":"172.21.15.6:6823/4008388154","state":["exists","up"]},{"osd":3,"uuid":"bd1c26cc-c121-4570-87ff-f432ecc53e6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":1341863884},{"type":"v1","addr":"172.21.15.6:6827","nonce":1341863884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":1341863884},{"type":"v1","addr":"172.21.15.6:6829","nonce":1341863884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":1341863884},{"type":"v1","addr":"172.21.15.6:6833","nonce":1341863884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":1341863884},{"type":"v1","addr":"172.21.15.6:6831","nonce":1341863884}]},"public_addr":"172.21.15.6:6827/1341863884","cluster_addr":"172.21.15.6:6829/1341863884","heartbeat_back_addr":"172.21.15.6:6833/1341863884","heartbeat_front_addr":"172.21.15.6:6831/1341863884","state":["exists","up"]},{"osd":4,"uuid":"65a98356-0941-41e8-a10b-9d0f76212fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3147977284},{"type":"v1","addr":"172.21.15.90:6801","nonce":3147977284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3147977284},{"type":"v1","addr":"172.21.15.90:6803","nonce":3147977284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3147977284},{"type":"v1","addr":"172.21.15.90:6807","nonce":3147977284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3147977284},{"type":"v1","addr":"172.21.15.90:6805","nonce":3147977284}]},"public_addr":"172.21.15.90:6801/3147977284","cluster_addr":"172.21.15.90:6803/3147977284","heartbeat_back_addr":"172.21.15.90:6807/3147977284","heartbeat_front_addr":"172.21.15.90:6805/3147977284","state":["exists","up"]},{"osd":5,"uuid":"b143b33c-9447-4b8c-9991-5037f50789ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":3585923902},{"type":"v1","addr":"172.21.15.90:6809","nonce":3585923902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3585923902},{"type":"v1","addr":"172.21.15.90:6811","nonce":3585923902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3585923902},{"type":"v1","addr":"172.21.15.90:6815","nonce":3585923902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3585923902},{"type":"v1","addr":"172.21.15.90:6813","nonce":3585923902}]},"public_addr":"172.21.15.90:6809/3585923902","cluster_addr":"172.21.15.90:6811/3585923902","heartbeat_back_addr":"172.21.15.90:6815/3585923902","heartbeat_front_addr":"172.21.15.90:6813/3585923902","state":["exists","up"]},{"osd":6,"uuid":"43226b7c-1700-47e4-8d1b-7f765e8defda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":713247979},{"type":"v1","addr":"172.21.15.90:6817","nonce":713247979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":713247979},{"type":"v1","addr":"172.21.15.90:6819","nonce":713247979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":713247979},{"type":"v1","addr":"172.21.15.90:6823","nonce":713247979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":713247979},{"type":"v1","addr":"172.21.15.90:6821","nonce":713247979}]},"public_addr":"172.21.15.90:6817/713247979","cluster_addr":"172.21.15.90:6819/713247979","heartbeat_back_addr":"172.21.15.90:6823/713247979","heartbeat_front_addr":"172.21.15.90:6821/713247979","state":["exists","up"]},{"osd":7,"uuid":"fa0def8c-eda9-441c-8ee9-e0021c064bf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1629523727},{"type":"v1","addr":"172.21.15.90:6825","nonce":1629523727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1629523727},{"type":"v1","addr":"172.21.15.90:6827","nonce":1629523727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1629523727},{"type":"v1","addr":"172.21.15.90:6831","nonce":1629523727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1629523727},{"type":"v1","addr":"172.21.15.90:6829","nonce":1629523727}]},"public_addr":"172.21.15.90:6825/1629523727","cluster_addr":"172.21.15.90:6827/1629523727","heartbeat_back_addr":"172.21.15.90:6831/1629523727","heartbeat_front_addr":"172.21.15.90:6829/1629523727","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:06:08.730123+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:06:33.621931+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:06:58.862101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:07:25.210912+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:07:49.154431+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:08:14.240635+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:08:39.558437+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-24T12:09:04.997129+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.6:0/1159789551":"2024-03-25T12:04:45.836851+0000","172.21.15.6:0/2683803593":"2024-03-25T12:04:45.836851+0000","172.21.15.6:6801/3010671792":"2024-03-25T12:04:45.836851+0000","172.21.15.6:6800/3010671792":"2024-03-25T12:04:45.836851+0000","172.21.15.6:6801/2054386689":"2024-03-25T12:04:18.792992+0000","172.21.15.6:6800/2054386689":"2024-03-25T12:04:18.792992+0000","172.21.15.6:0/897967172":"2024-03-25T12:04:18.792992+0000","172.21.15.6:0/2335021214":"2024-03-25T12:04:18.792992+0000","172.21.15.6:0/3268002752":"2024-03-25T12:04:18.792992+0000","172.21.15.6:0/1607559555":"2024-03-25T12:04:45.836851+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-24T12:09:57.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.960+0000 7ff5706b2700 1 -- 172.21.15.6:0/3955299007 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ff55406d650 msgr2=0x7ff55406fb10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-24T12:09:57.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.960+0000 7ff5706b2700 1 --2- 172.21.15.6:0/3955299007 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ff55406d650 0x7ff55406fb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:57.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.960+0000 7ff5706b2700 1 -- 172.21.15.6:0/3955299007 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff568100ba0 msgr2=0x7ff568102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:09:57.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.960+0000 7ff5706b2700 1 --2- 172.21.15.6:0/3955299007 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff568100ba0 0x7ff568102790 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff55800eb90 tx=0x7ff55800eea0 comp rx=0 tx=0).stop 2024-03-24T12:09:57.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.960+0000 7ff5706b2700 1 -- 172.21.15.6:0/3955299007 shutdown_connections 2024-03-24T12:09:57.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.960+0000 7ff5706b2700 1 --2- 172.21.15.6:0/3955299007 >> [v2:172.21.15.6:6800/3847865179,v1:172.21.15.6:6801/3847865179] conn(0x7ff55406d650 0x7ff55406fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:57.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.960+0000 7ff5706b2700 1 --2- 172.21.15.6:0/3955299007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff568106bd0 0x7ff568103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:57.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.960+0000 7ff5706b2700 1 --2- 172.21.15.6:0/3955299007 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff568100ba0 0x7ff568102790 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:57.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.960+0000 7ff5706b2700 1 --2- 172.21.15.6:0/3955299007 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff568101560 0x7ff568102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:09:57.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.960+0000 7ff5706b2700 1 -- 172.21.15.6:0/3955299007 >> 172.21.15.6:0/3955299007 conn(0x7ff568076bc0 msgr2=0x7ff56806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:09:57.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.960+0000 7ff5706b2700 1 -- 172.21.15.6:0/3955299007 shutdown_connections 2024-03-24T12:09:57.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:09:57.960+0000 7ff5706b2700 1 -- 172.21.15.6:0/3955299007 wait complete. 2024-03-24T12:09:58.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:09:58 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3955299007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T12:09:58.178 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:57 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:09:57.893+0000 7f05cbb92200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T12:09:58.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:09:58 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3955299007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T12:09:58.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:09:58 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3955299007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-24T12:09:58.211 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:09:57 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:09:57.919+0000 7f94c23d1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-24T12:09:58.211 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:09:58 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:09:58.116+0000 7f94c23d1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-24T12:09:58.450 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:58 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:09:58.186+0000 7f05cbb92200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T12:09:58.616 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:09:58 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:09:58.414+0000 7f94c23d1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-24T12:09:58.644 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-24T12:09:58.644 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-24T12:09:58.644 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-24T12:09:58.645 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-24T12:09:58.645 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-24T12:09:58.646 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-24T12:09:58.646 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-24T12:09:58.647 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-24T12:09:58.928 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:58 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:09:58.451+0000 7f05cbb92200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T12:09:58.928 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:58 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:09:58.691+0000 7f05cbb92200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T12:09:58.962 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:09:58 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:09:58.697+0000 7f94c23d1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-24T12:09:59.119 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:09:59.127 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:09:59.144 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:09:59.189 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:09:59.199 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:09:59.201 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:09:59.201 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:09:59.203 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:09:59.460 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:09:59 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:09:59.157+0000 7f94c23d1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-24T12:09:59.928 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:09:59 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:09:59.497+0000 7f05cbb92200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T12:10:00.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:00 smithi006 ceph-mon[30108]: overall HEALTH_OK 2024-03-24T12:10:00.211 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:10:00 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:10:00.044+0000 7f94c23d1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-24T12:10:00.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:00 smithi006 ceph-mon[41734]: overall HEALTH_OK 2024-03-24T12:10:00.379 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:00 smithi090 ceph-mon[33646]: overall HEALTH_OK 2024-03-24T12:10:00.379 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:10:00 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:10:00.063+0000 7f05cbb92200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T12:10:00.379 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:10:00 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:10:00.162+0000 7f05cbb92200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T12:10:00.678 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:10:00 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:10:00.379+0000 7f05cbb92200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T12:10:00.678 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:10:00 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: [24/Mar/2024:12:10:00] ENGINE Bus STARTING 2024-03-24T12:10:00.678 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:10:00 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: CherryPy Checker: 2024-03-24T12:10:00.679 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:10:00 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: The Application mounted at '' has an empty config. 2024-03-24T12:10:00.679 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:10:00 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: 2024-03-24T12:10:00.679 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:10:00 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: [24/Mar/2024:12:10:00] ENGINE Serving on http://:::9283 2024-03-24T12:10:00.679 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:10:00 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x[35716]: [24/Mar/2024:12:10:00] ENGINE Bus STARTED 2024-03-24T12:10:00.915 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:10:00 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:10:00.641+0000 7f94c23d1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-24T12:10:00.915 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:10:00 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:10:00.744+0000 7f94c23d1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-24T12:10:01.198 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:10:00 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:10:00.975+0000 7f94c23d1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-24T12:10:01.199 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:10:01] ENGINE Bus STARTING 2024-03-24T12:10:01.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: Standby manager daemon x restarted 2024-03-24T12:10:01.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: Standby manager daemon x started 2024-03-24T12:10:01.199 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: from='mgr.? 172.21.15.90:0/3155380163' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T12:10:01.200 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: from='mgr.? 172.21.15.90:0/3155380163' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T12:10:01.200 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: from='mgr.? 172.21.15.90:0/3155380163' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T12:10:01.200 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: from='mgr.? 172.21.15.90:0/3155380163' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T12:10:01.200 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: Active manager daemon y restarted 2024-03-24T12:10:01.200 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: Activating manager daemon y 2024-03-24T12:10:01.200 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: osdmap e72: 8 total, 8 up, 8 in 2024-03-24T12:10:01.200 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: mgrmap e18: y(active, starting, since 0.00881065s), standbys: x 2024-03-24T12:10:01.200 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T12:10:01.200 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:10:01.200 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:10:01.201 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T12:10:01.201 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T12:10:01.201 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T12:10:01.201 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T12:10:01.201 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T12:10:01.201 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T12:10:01.201 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T12:10:01.201 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T12:10:01.201 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T12:10:01.201 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T12:10:01.201 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T12:10:01.201 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T12:10:01.201 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T12:10:01.201 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[30108]: Manager daemon y is now available 2024-03-24T12:10:01.202 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: Standby manager daemon x restarted 2024-03-24T12:10:01.202 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: Standby manager daemon x started 2024-03-24T12:10:01.202 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: from='mgr.? 172.21.15.90:0/3155380163' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T12:10:01.202 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: from='mgr.? 172.21.15.90:0/3155380163' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T12:10:01.202 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: from='mgr.? 172.21.15.90:0/3155380163' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T12:10:01.202 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: from='mgr.? 172.21.15.90:0/3155380163' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T12:10:01.202 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: Active manager daemon y restarted 2024-03-24T12:10:01.202 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: Activating manager daemon y 2024-03-24T12:10:01.202 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: osdmap e72: 8 total, 8 up, 8 in 2024-03-24T12:10:01.202 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: mgrmap e18: y(active, starting, since 0.00881065s), standbys: x 2024-03-24T12:10:01.202 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T12:10:01.203 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:10:01.203 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:10:01.203 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T12:10:01.203 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T12:10:01.203 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T12:10:01.203 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T12:10:01.203 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T12:10:01.203 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T12:10:01.203 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T12:10:01.203 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T12:10:01.203 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T12:10:01.203 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T12:10:01.203 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T12:10:01.203 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T12:10:01.204 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T12:10:01.204 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-mon[41734]: Manager daemon y is now available 2024-03-24T12:10:01.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: Standby manager daemon x restarted 2024-03-24T12:10:01.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: Standby manager daemon x started 2024-03-24T12:10:01.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: from='mgr.? 172.21.15.90:0/3155380163' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-24T12:10:01.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: from='mgr.? 172.21.15.90:0/3155380163' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-24T12:10:01.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: from='mgr.? 172.21.15.90:0/3155380163' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-24T12:10:01.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: from='mgr.? 172.21.15.90:0/3155380163' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-24T12:10:01.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: Active manager daemon y restarted 2024-03-24T12:10:01.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: Activating manager daemon y 2024-03-24T12:10:01.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: osdmap e72: 8 total, 8 up, 8 in 2024-03-24T12:10:01.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: mgrmap e18: y(active, starting, since 0.00881065s), standbys: x 2024-03-24T12:10:01.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-24T12:10:01.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-24T12:10:01.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-24T12:10:01.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-24T12:10:01.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-24T12:10:01.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-24T12:10:01.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-24T12:10:01.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-24T12:10:01.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-24T12:10:01.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-24T12:10:01.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-24T12:10:01.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-24T12:10:01.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-24T12:10:01.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-24T12:10:01.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-24T12:10:01.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-24T12:10:01.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:01 smithi090 ceph-mon[33646]: Manager daemon y is now available 2024-03-24T12:10:01.461 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: CherryPy Checker: 2024-03-24T12:10:01.461 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: The Application mounted at '' has an empty config. 2024-03-24T12:10:01.461 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: 2024-03-24T12:10:01.461 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:10:01] ENGINE Serving on http://:::9283 2024-03-24T12:10:01.461 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:10:01 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:10:01] ENGINE Bus STARTED 2024-03-24T12:10:02.167 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:02 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:10:02.167 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:02 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:10:02.167 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:02 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:02.167 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:02 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:10:02.167 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:02 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T12:10:02.167 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:02 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T12:10:02.167 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:02 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T12:10:02.168 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:02 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T12:10:02.168 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:02 smithi006 ceph-mon[30108]: mgrmap e19: y(active, since 1.01159s), standbys: x 2024-03-24T12:10:02.168 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:02 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:10:02.168 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:02 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:10:02.168 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:02 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:02.168 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:02 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:10:02.168 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:02 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T12:10:02.168 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:02 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T12:10:02.168 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:02 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T12:10:02.168 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:02 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T12:10:02.168 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:02 smithi006 ceph-mon[41734]: mgrmap e19: y(active, since 1.01159s), standbys: x 2024-03-24T12:10:02.366 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:10:02 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:10:02.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:02 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:10:02.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:02 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:10:02.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:02 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:02.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:02 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:10:02.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:02 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T12:10:02.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:02 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-24T12:10:02.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:02 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T12:10:02.366 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:02 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-24T12:10:02.367 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:02 smithi090 ceph-mon[33646]: mgrmap e19: y(active, since 1.01159s), standbys: x 2024-03-24T12:10:03.308 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:03 smithi006 ceph-mon[30108]: [24/Mar/2024:12:10:01] ENGINE Bus STARTING 2024-03-24T12:10:03.309 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:03 smithi006 ceph-mon[30108]: [24/Mar/2024:12:10:01] ENGINE Serving on https://172.21.15.6:7150 2024-03-24T12:10:03.309 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:03 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:10:03.309 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:03 smithi006 ceph-mon[30108]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:10:03.309 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:03 smithi006 ceph-mon[30108]: [24/Mar/2024:12:10:02] ENGINE Serving on http://172.21.15.6:8765 2024-03-24T12:10:03.309 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:03 smithi006 ceph-mon[30108]: [24/Mar/2024:12:10:02] ENGINE Bus STARTED 2024-03-24T12:10:03.309 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:03 smithi006 ceph-mon[41734]: [24/Mar/2024:12:10:01] ENGINE Bus STARTING 2024-03-24T12:10:03.309 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:03 smithi006 ceph-mon[41734]: [24/Mar/2024:12:10:01] ENGINE Serving on https://172.21.15.6:7150 2024-03-24T12:10:03.309 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:03 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:10:03.309 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:03 smithi006 ceph-mon[41734]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:10:03.310 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:03 smithi006 ceph-mon[41734]: [24/Mar/2024:12:10:02] ENGINE Serving on http://172.21.15.6:8765 2024-03-24T12:10:03.310 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:03 smithi006 ceph-mon[41734]: [24/Mar/2024:12:10:02] ENGINE Bus STARTED 2024-03-24T12:10:03.410 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:03 smithi090 ceph-mon[33646]: [24/Mar/2024:12:10:01] ENGINE Bus STARTING 2024-03-24T12:10:03.410 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:03 smithi090 ceph-mon[33646]: [24/Mar/2024:12:10:01] ENGINE Serving on https://172.21.15.6:7150 2024-03-24T12:10:03.410 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:03 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:10:03.410 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:03 smithi090 ceph-mon[33646]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:10:03.411 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:03 smithi090 ceph-mon[33646]: [24/Mar/2024:12:10:02] ENGINE Serving on http://172.21.15.6:8765 2024-03-24T12:10:03.411 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:03 smithi090 ceph-mon[33646]: [24/Mar/2024:12:10:02] ENGINE Bus STARTED 2024-03-24T12:10:04.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:04 smithi006 ceph-mon[30108]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:10:04.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:04 smithi006 ceph-mon[30108]: mgrmap e20: y(active, since 3s), standbys: x 2024-03-24T12:10:04.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:04 smithi006 ceph-mon[41734]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:10:04.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:04 smithi006 ceph-mon[41734]: mgrmap e20: y(active, since 3s), standbys: x 2024-03-24T12:10:04.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:04 smithi090 ceph-mon[33646]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:10:04.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:04 smithi090 ceph-mon[33646]: mgrmap e20: y(active, since 3s), standbys: x 2024-03-24T12:10:05.568 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:05 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:05.568 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:05 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:05.568 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:05 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:05.568 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:05 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:05.568 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:10:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:10:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:10:05.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:05 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:05.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:05 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:06.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.438+0000 7f6a4e248700 1 -- 172.21.15.6:0/48268541 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a48106bd0 msgr2=0x7f6a48106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:06.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.438+0000 7f6a4e248700 1 --2- 172.21.15.6:0/48268541 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a48106bd0 0x7f6a48106fb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c009990 tx=0x7f6a3c009ca0 comp rx=0 tx=0).stop 2024-03-24T12:10:06.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.438+0000 7f6a4e248700 1 -- 172.21.15.6:0/48268541 shutdown_connections 2024-03-24T12:10:06.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.438+0000 7f6a4e248700 1 --2- 172.21.15.6:0/48268541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a48101560 0x7f6a4810d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:06.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.438+0000 7f6a4e248700 1 --2- 172.21.15.6:0/48268541 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a48106bd0 0x7f6a48106fb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:06.531 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.438+0000 7f6a4e248700 1 --2- 172.21.15.6:0/48268541 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6a48100ba0 0x7f6a48101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:06.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.438+0000 7f6a4e248700 1 -- 172.21.15.6:0/48268541 >> 172.21.15.6:0/48268541 conn(0x7f6a48076bc0 msgr2=0x7f6a48076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:06.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.438+0000 7f6a4e248700 1 -- 172.21.15.6:0/48268541 shutdown_connections 2024-03-24T12:10:06.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.438+0000 7f6a4e248700 1 -- 172.21.15.6:0/48268541 wait complete. 2024-03-24T12:10:06.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.439+0000 7f6a4e248700 1 Processor -- start 2024-03-24T12:10:06.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.439+0000 7f6a4e248700 1 -- start start 2024-03-24T12:10:06.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.439+0000 7f6a4e248700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6a48100ba0 0x7f6a481965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:06.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.439+0000 7f6a4e248700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a48101560 0x7f6a48196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:06.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.439+0000 7f6a4e248700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a48106bd0 0x7f6a4819ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:06.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.439+0000 7f6a4e248700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4819b4a0 con 0x7f6a48101560 2024-03-24T12:10:06.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.439+0000 7f6a4e248700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4819b610 con 0x7f6a48106bd0 2024-03-24T12:10:06.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.439+0000 7f6a4e248700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4819b7d0 con 0x7f6a48100ba0 2024-03-24T12:10:06.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.440+0000 7f6a477fe700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6a48100ba0 0x7f6a481965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:06.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.440+0000 7f6a477fe700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6a48100ba0 0x7f6a481965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:35254/0 (socket says 172.21.15.6:35254) 2024-03-24T12:10:06.532 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.440+0000 7f6a477fe700 1 -- 172.21.15.6:0/1543147079 learned_addr learned my addr 172.21.15.6:0/1543147079 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:10:06.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.440+0000 7f6a46ffd700 1 --2- 172.21.15.6:0/1543147079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a48101560 0x7f6a48196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:06.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.440+0000 7f6a47fff700 1 --2- 172.21.15.6:0/1543147079 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a48106bd0 0x7f6a4819ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:06.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.440+0000 7f6a477fe700 1 -- 172.21.15.6:0/1543147079 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a48106bd0 msgr2=0x7f6a4819ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:06.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.440+0000 7f6a477fe700 1 --2- 172.21.15.6:0/1543147079 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a48106bd0 0x7f6a4819ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:06.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.440+0000 7f6a477fe700 1 -- 172.21.15.6:0/1543147079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a48101560 msgr2=0x7f6a48196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:06.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.440+0000 7f6a477fe700 1 --2- 172.21.15.6:0/1543147079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a48101560 0x7f6a48196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:06.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.440+0000 7f6a477fe700 1 -- 172.21.15.6:0/1543147079 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3c0095f0 con 0x7f6a48100ba0 2024-03-24T12:10:06.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.440+0000 7f6a477fe700 1 --2- 172.21.15.6:0/1543147079 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6a48100ba0 0x7f6a481965c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c003f50 tx=0x7f6a3c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:06.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.440+0000 7f6a44ff9700 1 -- 172.21.15.6:0/1543147079 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c01d070 con 0x7f6a48100ba0 2024-03-24T12:10:06.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.440+0000 7f6a44ff9700 1 -- 172.21.15.6:0/1543147079 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f6a3c00bd20 con 0x7f6a48100ba0 2024-03-24T12:10:06.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.440+0000 7f6a4e248700 1 -- 172.21.15.6:0/1543147079 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a4819ba50 con 0x7f6a48100ba0 2024-03-24T12:10:06.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.440+0000 7f6a44ff9700 1 -- 172.21.15.6:0/1543147079 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c022d30 con 0x7f6a48100ba0 2024-03-24T12:10:06.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.440+0000 7f6a4e248700 1 -- 172.21.15.6:0/1543147079 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a481a3310 con 0x7f6a48100ba0 2024-03-24T12:10:06.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.441+0000 7f6a4e248700 1 -- 172.21.15.6:0/1543147079 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6a48110390 con 0x7f6a48100ba0 2024-03-24T12:10:06.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.442+0000 7f6a44ff9700 1 -- 172.21.15.6:0/1543147079 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7f6a3c022470 con 0x7f6a48100ba0 2024-03-24T12:10:06.533 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.442+0000 7f6a44ff9700 1 --2- 172.21.15.6:0/1543147079 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f6a3006d660 0x7f6a3006fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:06.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.442+0000 7f6a44ff9700 1 -- 172.21.15.6:0/1543147079 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f6a3c08f350 con 0x7f6a48100ba0 2024-03-24T12:10:06.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.442+0000 7f6a46ffd700 1 --2- 172.21.15.6:0/1543147079 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f6a3006d660 0x7f6a3006fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:06.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.443+0000 7f6a44ff9700 1 --2- 172.21.15.6:0/1543147079 >> [v2:172.21.15.90:6800/3147977284,v1:172.21.15.90:6801/3147977284] conn(0x7f6a30077570 0x7f6a30079a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:06.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.443+0000 7f6a44ff9700 1 -- 172.21.15.6:0/1543147079 --> [v2:172.21.15.90:6800/3147977284,v1:172.21.15.90:6801/3147977284] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6a3007a0e0 con 0x7f6a30077570 2024-03-24T12:10:06.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.443+0000 7f6a44ff9700 1 -- 172.21.15.6:0/1543147079 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a3c08f7a0 con 0x7f6a48100ba0 2024-03-24T12:10:06.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.443+0000 7f6a47fff700 1 --2- 172.21.15.6:0/1543147079 >> [v2:172.21.15.90:6800/3147977284,v1:172.21.15.90:6801/3147977284] conn(0x7f6a30077570 0x7f6a30079a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:06.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.443+0000 7f6a46ffd700 1 --2- 172.21.15.6:0/1543147079 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f6a3006d660 0x7f6a3006fb20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6a48077370 tx=0x7f6a38009260 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:06.534 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.443+0000 7f6a47fff700 1 --2- 172.21.15.6:0/1543147079 >> [v2:172.21.15.90:6800/3147977284,v1:172.21.15.90:6801/3147977284] conn(0x7f6a30077570 0x7f6a30079a10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:06.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.444+0000 7f6a44ff9700 1 -- 172.21.15.6:0/1543147079 <== osd.4 v2:172.21.15.90:6800/3147977284 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6a3007a0e0 con 0x7f6a30077570 2024-03-24T12:10:06.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.461+0000 7f6a4e248700 1 -- 172.21.15.6:0/1543147079 --> [v2:172.21.15.90:6800/3147977284,v1:172.21.15.90:6801/3147977284] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6a48108220 con 0x7f6a30077570 2024-03-24T12:10:06.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.462+0000 7f6a44ff9700 1 -- 172.21.15.6:0/1543147079 <== osd.4 v2:172.21.15.90:6800/3147977284 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6a48108220 con 0x7f6a30077570 2024-03-24T12:10:06.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.462+0000 7f6a4e248700 1 -- 172.21.15.6:0/1543147079 >> [v2:172.21.15.90:6800/3147977284,v1:172.21.15.90:6801/3147977284] conn(0x7f6a30077570 msgr2=0x7f6a30079a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:06.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.462+0000 7f6a4e248700 1 --2- 172.21.15.6:0/1543147079 >> [v2:172.21.15.90:6800/3147977284,v1:172.21.15.90:6801/3147977284] conn(0x7f6a30077570 0x7f6a30079a10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:06.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.462+0000 7f6a4e248700 1 -- 172.21.15.6:0/1543147079 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f6a3006d660 msgr2=0x7f6a3006fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:06.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.462+0000 7f6a4e248700 1 --2- 172.21.15.6:0/1543147079 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f6a3006d660 0x7f6a3006fb20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6a48077370 tx=0x7f6a38009260 comp rx=0 tx=0).stop 2024-03-24T12:10:06.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.462+0000 7f6a4e248700 1 -- 172.21.15.6:0/1543147079 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6a48100ba0 msgr2=0x7f6a481965c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:06.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.462+0000 7f6a4e248700 1 --2- 172.21.15.6:0/1543147079 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6a48100ba0 0x7f6a481965c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c003f50 tx=0x7f6a3c005740 comp rx=0 tx=0).stop 2024-03-24T12:10:06.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.462+0000 7f6a47fff700 1 -- 172.21.15.6:0/1543147079 reap_dead start 2024-03-24T12:10:06.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.462+0000 7f6a4e248700 1 -- 172.21.15.6:0/1543147079 shutdown_connections 2024-03-24T12:10:06.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.462+0000 7f6a4e248700 1 -- 172.21.15.6:0/1543147079 >> 172.21.15.6:0/1543147079 conn(0x7f6a48076bc0 msgr2=0x7f6a481078c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:06.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.463+0000 7f6a4e248700 1 -- 172.21.15.6:0/1543147079 shutdown_connections 2024-03-24T12:10:06.535 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.463+0000 7f6a4e248700 1 -- 172.21.15.6:0/1543147079 wait complete. 2024-03-24T12:10:06.902 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[30108]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:10:06.902 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[30108]: mgrmap e21: y(active, since 4s), standbys: x 2024-03-24T12:10:06.902 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:06.902 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:06.902 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.902 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.902 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.902 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.902 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.903 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.903 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.903 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.903 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[30108]: Adjusting osd_memory_target on smithi090 to 3760M 2024-03-24T12:10:06.903 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:06.903 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[41734]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:10:06.903 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[41734]: mgrmap e21: y(active, since 4s), standbys: x 2024-03-24T12:10:06.903 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:06.903 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:06.903 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.903 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.903 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.904 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.904 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.904 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.904 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.904 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.904 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[41734]: Adjusting osd_memory_target on smithi090 to 3760M 2024-03-24T12:10:06.904 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:06 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:06.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:06 smithi090 ceph-mon[33646]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:10:06.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:06 smithi090 ceph-mon[33646]: mgrmap e21: y(active, since 4s), standbys: x 2024-03-24T12:10:06.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:06 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:06.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:06 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:06.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:06 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:06 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:06 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:06 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:06 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:06 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:06 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:06 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:06.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:06 smithi090 ceph-mon[33646]: Adjusting osd_memory_target on smithi090 to 3760M 2024-03-24T12:10:06.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:06 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:08.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.525+0000 7fdeab594700 1 -- 172.21.15.6:0/778661531 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdea4100c20 msgr2=0x7fdea410eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:08.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.525+0000 7fdeab594700 1 --2- 172.21.15.6:0/778661531 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdea4100c20 0x7fdea410eb80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fde9c00b0a0 tx=0x7fde9c00b3b0 comp rx=0 tx=0).stop 2024-03-24T12:10:08.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.525+0000 7fdeab594700 1 -- 172.21.15.6:0/778661531 shutdown_connections 2024-03-24T12:10:08.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.525+0000 7fdeab594700 1 --2- 172.21.15.6:0/778661531 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdea4100c20 0x7fdea410eb80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:08.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.525+0000 7fdeab594700 1 --2- 172.21.15.6:0/778661531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdea4106ce0 0x7fdea41070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:08.227 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.525+0000 7fdeab594700 1 --2- 172.21.15.6:0/778661531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdea4100260 0x7fdea41006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:08.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.525+0000 7fdeab594700 1 -- 172.21.15.6:0/778661531 >> 172.21.15.6:0/778661531 conn(0x7fdea4077170 msgr2=0x7fdea40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:08.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.525+0000 7fdeab594700 1 -- 172.21.15.6:0/778661531 shutdown_connections 2024-03-24T12:10:08.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.525+0000 7fdeab594700 1 -- 172.21.15.6:0/778661531 wait complete. 2024-03-24T12:10:08.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.525+0000 7fdeab594700 1 Processor -- start 2024-03-24T12:10:08.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.525+0000 7fdeab594700 1 -- start start 2024-03-24T12:10:08.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.526+0000 7fdeab594700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdea4100260 0x7fdea4101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:08.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.526+0000 7fdeab594700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdea4100c20 0x7fdea4101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:08.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.526+0000 7fdeab594700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdea4106ce0 0x7fdea4101ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:08.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.526+0000 7fdeab594700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea41a5d70 con 0x7fdea4100260 2024-03-24T12:10:08.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.526+0000 7fdeab594700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea41a5ee0 con 0x7fdea4100c20 2024-03-24T12:10:08.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.526+0000 7fdeab594700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea41a60a0 con 0x7fdea4106ce0 2024-03-24T12:10:08.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.526+0000 7fdea9330700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdea4100260 0x7fdea4101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:08.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.526+0000 7fdea9b31700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdea4106ce0 0x7fdea4101ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:08.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.526+0000 7fdea9330700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdea4100260 0x7fdea4101440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49728/0 (socket says 172.21.15.6:49728) 2024-03-24T12:10:08.228 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.526+0000 7fdea9330700 1 -- 172.21.15.6:0/3651740786 learned_addr learned my addr 172.21.15.6:0/3651740786 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:10:08.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.526+0000 7fdea8b2f700 1 --2- 172.21.15.6:0/3651740786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdea4100c20 0x7fdea4101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:08.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.526+0000 7fdea9330700 1 -- 172.21.15.6:0/3651740786 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdea4106ce0 msgr2=0x7fdea4101ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:08.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.526+0000 7fdea9330700 1 --2- 172.21.15.6:0/3651740786 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdea4106ce0 0x7fdea4101ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:08.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.526+0000 7fdea9330700 1 -- 172.21.15.6:0/3651740786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdea4100c20 msgr2=0x7fdea4101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:08.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.526+0000 7fdea9330700 1 --2- 172.21.15.6:0/3651740786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdea4100c20 0x7fdea4101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:08.229 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.526+0000 7fdea9330700 1 -- 172.21.15.6:0/3651740786 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea0009520 con 0x7fdea4100260 2024-03-24T12:10:08.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.526+0000 7fdea9b31700 1 --2- 172.21.15.6:0/3651740786 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdea4106ce0 0x7fdea4101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:10:08.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.526+0000 7fdea9330700 1 --2- 172.21.15.6:0/3651740786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdea4100260 0x7fdea4101440 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdea000eb30 tx=0x7fdea000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:08.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.526+0000 7fde9a7fc700 1 -- 172.21.15.6:0/3651740786 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea0009880 con 0x7fdea4100260 2024-03-24T12:10:08.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.526+0000 7fde9a7fc700 1 -- 172.21.15.6:0/3651740786 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fdea00044a0 con 0x7fdea4100260 2024-03-24T12:10:08.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.527+0000 7fdeab594700 1 -- 172.21.15.6:0/3651740786 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde9c009d00 con 0x7fdea4100260 2024-03-24T12:10:08.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.527+0000 7fde9a7fc700 1 -- 172.21.15.6:0/3651740786 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea0003ca0 con 0x7fdea4100260 2024-03-24T12:10:08.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.527+0000 7fdeab594700 1 -- 172.21.15.6:0/3651740786 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea41a6740 con 0x7fdea4100260 2024-03-24T12:10:08.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.527+0000 7fdeab594700 1 -- 172.21.15.6:0/3651740786 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdea41115e0 con 0x7fdea4100260 2024-03-24T12:10:08.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.528+0000 7fde9a7fc700 1 -- 172.21.15.6:0/3651740786 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7fdea0003e00 con 0x7fdea4100260 2024-03-24T12:10:08.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.528+0000 7fde9a7fc700 1 --2- 172.21.15.6:0/3651740786 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fde9006d660 0x7fde9006fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:08.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.528+0000 7fde9a7fc700 1 -- 172.21.15.6:0/3651740786 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7fdea0014070 con 0x7fdea4100260 2024-03-24T12:10:08.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.528+0000 7fdea8b2f700 1 --2- 172.21.15.6:0/3651740786 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fde9006d660 0x7fde9006fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:08.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.529+0000 7fde9a7fc700 1 --2- 172.21.15.6:0/3651740786 >> [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154] conn(0x7fde90077570 0x7fde90079a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:08.230 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.529+0000 7fde9a7fc700 1 -- 172.21.15.6:0/3651740786 --> [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fde9007a0e0 con 0x7fde90077570 2024-03-24T12:10:08.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.529+0000 7fde9a7fc700 1 -- 172.21.15.6:0/3651740786 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdea008dcc0 con 0x7fdea4100260 2024-03-24T12:10:08.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.529+0000 7fdea9b31700 1 --2- 172.21.15.6:0/3651740786 >> [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154] conn(0x7fde90077570 0x7fde90079a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:08.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.529+0000 7fdea8b2f700 1 --2- 172.21.15.6:0/3651740786 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fde9006d660 0x7fde9006fb20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdea40ff810 tx=0x7fde94009260 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:08.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.529+0000 7fdea9b31700 1 --2- 172.21.15.6:0/3651740786 >> [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154] conn(0x7fde90077570 0x7fde90079a10 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:08.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.530+0000 7fde9a7fc700 1 -- 172.21.15.6:0/3651740786 <== osd.2 v2:172.21.15.6:6818/4008388154 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fde9007a0e0 con 0x7fde90077570 2024-03-24T12:10:08.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.546+0000 7fdeab594700 1 -- 172.21.15.6:0/3651740786 --> [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdea4068930 con 0x7fde90077570 2024-03-24T12:10:08.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.547+0000 7fde9a7fc700 1 -- 172.21.15.6:0/3651740786 <== osd.2 v2:172.21.15.6:6818/4008388154 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdea4068930 con 0x7fde90077570 2024-03-24T12:10:08.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.547+0000 7fdeab594700 1 -- 172.21.15.6:0/3651740786 >> [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154] conn(0x7fde90077570 msgr2=0x7fde90079a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:08.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.547+0000 7fdeab594700 1 --2- 172.21.15.6:0/3651740786 >> [v2:172.21.15.6:6818/4008388154,v1:172.21.15.6:6819/4008388154] conn(0x7fde90077570 0x7fde90079a10 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:08.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.547+0000 7fdeab594700 1 -- 172.21.15.6:0/3651740786 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fde9006d660 msgr2=0x7fde9006fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:08.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.547+0000 7fdeab594700 1 --2- 172.21.15.6:0/3651740786 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fde9006d660 0x7fde9006fb20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdea40ff810 tx=0x7fde94009260 comp rx=0 tx=0).stop 2024-03-24T12:10:08.231 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.547+0000 7fdeab594700 1 -- 172.21.15.6:0/3651740786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdea4100260 msgr2=0x7fdea4101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:08.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.547+0000 7fdeab594700 1 --2- 172.21.15.6:0/3651740786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdea4100260 0x7fdea4101440 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdea000eb30 tx=0x7fdea000ee40 comp rx=0 tx=0).stop 2024-03-24T12:10:08.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.547+0000 7fdea9b31700 1 -- 172.21.15.6:0/3651740786 reap_dead start 2024-03-24T12:10:08.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.547+0000 7fdeab594700 1 -- 172.21.15.6:0/3651740786 shutdown_connections 2024-03-24T12:10:08.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.547+0000 7fdeab594700 1 -- 172.21.15.6:0/3651740786 >> 172.21.15.6:0/3651740786 conn(0x7fdea4077170 msgr2=0x7fdea41047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:08.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.547+0000 7fdeab594700 1 -- 172.21.15.6:0/3651740786 shutdown_connections 2024-03-24T12:10:08.232 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:06.547+0000 7fdeab594700 1 -- 172.21.15.6:0/3651740786 wait complete. 2024-03-24T12:10:08.839 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:08 smithi006 ceph-mon[30108]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 24 op/s 2024-03-24T12:10:08.839 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:08 smithi006 ceph-mon[41734]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 24 op/s 2024-03-24T12:10:08.867 INFO:teuthology.orchestra.run.smithi006.stdout:77309411367 2024-03-24T12:10:08.867 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-24T12:10:08.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:08 smithi090 ceph-mon[33646]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 24 op/s 2024-03-24T12:10:09.439 INFO:teuthology.orchestra.run.smithi006.stdout:128849018908 2024-03-24T12:10:09.440 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-24T12:10:09.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.919+0000 7f2792891700 1 -- 172.21.15.6:0/3781291826 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f278c1012d0 msgr2=0x7f278c101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:09.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.919+0000 7f2792891700 1 --2- 172.21.15.6:0/3781291826 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f278c1012d0 0x7f278c101750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f27800098f0 tx=0x7f2780009c00 comp rx=0 tx=0).stop 2024-03-24T12:10:09.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.919+0000 7f2792891700 1 -- 172.21.15.6:0/3781291826 shutdown_connections 2024-03-24T12:10:09.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.919+0000 7f2792891700 1 --2- 172.21.15.6:0/3781291826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f278c101c90 0x7f278c10d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:09.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.919+0000 7f2792891700 1 --2- 172.21.15.6:0/3781291826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f278c105b20 0x7f278c105f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:09.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.919+0000 7f2792891700 1 --2- 172.21.15.6:0/3781291826 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f278c1012d0 0x7f278c101750 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:09.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.919+0000 7f2792891700 1 -- 172.21.15.6:0/3781291826 >> 172.21.15.6:0/3781291826 conn(0x7f278c077140 msgr2=0x7f278c0fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:09.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.919+0000 7f2792891700 1 -- 172.21.15.6:0/3781291826 shutdown_connections 2024-03-24T12:10:09.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.919+0000 7f2792891700 1 -- 172.21.15.6:0/3781291826 wait complete. 2024-03-24T12:10:09.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.920+0000 7f2792891700 1 Processor -- start 2024-03-24T12:10:09.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.920+0000 7f2792891700 1 -- start start 2024-03-24T12:10:09.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.920+0000 7f2792891700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f278c1012d0 0x7f278c0fc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:09.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.920+0000 7f2792891700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f278c101c90 0x7f278c0fcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:09.920 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.920+0000 7f2792891700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f278c105b20 0x7f278c0fd030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:09.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.920+0000 7f2792891700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c1a5c00 con 0x7f278c105b20 2024-03-24T12:10:09.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.920+0000 7f2792891700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c1a5d70 con 0x7f278c101c90 2024-03-24T12:10:09.921 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.920+0000 7f2792891700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278c1a5f30 con 0x7f278c1012d0 2024-03-24T12:10:09.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.921+0000 7f2790e2e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f278c105b20 0x7f278c0fd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:09.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.921+0000 7f278bfff700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f278c1012d0 0x7f278c0fc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:09.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.921+0000 7f2790e2e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f278c105b20 0x7f278c0fd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49744/0 (socket says 172.21.15.6:49744) 2024-03-24T12:10:09.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.921+0000 7f2790e2e700 1 -- 172.21.15.6:0/2596175156 learned_addr learned my addr 172.21.15.6:0/2596175156 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:10:09.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.921+0000 7f278b7fe700 1 --2- 172.21.15.6:0/2596175156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f278c101c90 0x7f278c0fcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:09.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.921+0000 7f2790e2e700 1 -- 172.21.15.6:0/2596175156 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f278c1012d0 msgr2=0x7f278c0fc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:09.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.921+0000 7f2790e2e700 1 --2- 172.21.15.6:0/2596175156 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f278c1012d0 0x7f278c0fc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:09.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.921+0000 7f2790e2e700 1 -- 172.21.15.6:0/2596175156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f278c101c90 msgr2=0x7f278c0fcaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:09.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.921+0000 7f2790e2e700 1 --2- 172.21.15.6:0/2596175156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f278c101c90 0x7f278c0fcaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:09.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.921+0000 7f2790e2e700 1 -- 172.21.15.6:0/2596175156 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277c0095f0 con 0x7f278c105b20 2024-03-24T12:10:09.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.921+0000 7f2790e2e700 1 --2- 172.21.15.6:0/2596175156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f278c105b20 0x7f278c0fd030 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2784007ee0 tx=0x7f278400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:09.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.921+0000 7f27897fa700 1 -- 172.21.15.6:0/2596175156 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2784017070 con 0x7f278c105b20 2024-03-24T12:10:09.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.921+0000 7f2792891700 1 -- 172.21.15.6:0/2596175156 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2780009520 con 0x7f278c105b20 2024-03-24T12:10:09.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.921+0000 7f27897fa700 1 -- 172.21.15.6:0/2596175156 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f278400f040 con 0x7f278c105b20 2024-03-24T12:10:09.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.921+0000 7f27897fa700 1 -- 172.21.15.6:0/2596175156 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27840054a0 con 0x7f278c105b20 2024-03-24T12:10:09.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.921+0000 7f2792891700 1 -- 172.21.15.6:0/2596175156 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278c1a6550 con 0x7f278c105b20 2024-03-24T12:10:09.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.922+0000 7f2792891700 1 -- 172.21.15.6:0/2596175156 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f278c110480 con 0x7f278c105b20 2024-03-24T12:10:09.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.923+0000 7f27897fa700 1 -- 172.21.15.6:0/2596175156 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7f278400db20 con 0x7f278c105b20 2024-03-24T12:10:09.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.923+0000 7f27897fa700 1 --2- 172.21.15.6:0/2596175156 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f277406d660 0x7f277406fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:09.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.923+0000 7f278bfff700 1 --2- 172.21.15.6:0/2596175156 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f277406d660 0x7f277406fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:09.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.923+0000 7f27897fa700 1 -- 172.21.15.6:0/2596175156 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f278408cab0 con 0x7f278c105b20 2024-03-24T12:10:09.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.923+0000 7f27897fa700 1 --2- 172.21.15.6:0/2596175156 >> [v2:172.21.15.90:6816/713247979,v1:172.21.15.90:6817/713247979] conn(0x7f27740774f0 0x7f2774079990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:09.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.924+0000 7f27897fa700 1 -- 172.21.15.6:0/2596175156 --> [v2:172.21.15.90:6816/713247979,v1:172.21.15.90:6817/713247979] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f277407a060 con 0x7f27740774f0 2024-03-24T12:10:09.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.924+0000 7f27897fa700 1 -- 172.21.15.6:0/2596175156 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27840912a0 con 0x7f278c105b20 2024-03-24T12:10:09.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.924+0000 7f278bfff700 1 --2- 172.21.15.6:0/2596175156 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f277406d660 0x7f277406fb20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f277c005fd0 tx=0x7f277c009310 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:09.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.924+0000 7f278b7fe700 1 --2- 172.21.15.6:0/2596175156 >> [v2:172.21.15.90:6816/713247979,v1:172.21.15.90:6817/713247979] conn(0x7f27740774f0 0x7f2774079990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:09.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.924+0000 7f278b7fe700 1 --2- 172.21.15.6:0/2596175156 >> [v2:172.21.15.90:6816/713247979,v1:172.21.15.90:6817/713247979] conn(0x7f27740774f0 0x7f2774079990 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:09.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.925+0000 7f27897fa700 1 -- 172.21.15.6:0/2596175156 <== osd.6 v2:172.21.15.90:6816/713247979 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f277407a060 con 0x7f27740774f0 2024-03-24T12:10:09.940 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.941+0000 7f2792891700 1 -- 172.21.15.6:0/2596175156 --> [v2:172.21.15.90:6816/713247979,v1:172.21.15.90:6817/713247979] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f278c0fe920 con 0x7f27740774f0 2024-03-24T12:10:09.941 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.942+0000 7f27897fa700 1 -- 172.21.15.6:0/2596175156 <== osd.6 v2:172.21.15.90:6816/713247979 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f278c0fe920 con 0x7f27740774f0 2024-03-24T12:10:09.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.942+0000 7f2792891700 1 -- 172.21.15.6:0/2596175156 >> [v2:172.21.15.90:6816/713247979,v1:172.21.15.90:6817/713247979] conn(0x7f27740774f0 msgr2=0x7f2774079990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:09.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.942+0000 7f2792891700 1 --2- 172.21.15.6:0/2596175156 >> [v2:172.21.15.90:6816/713247979,v1:172.21.15.90:6817/713247979] conn(0x7f27740774f0 0x7f2774079990 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:09.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.942+0000 7f2792891700 1 -- 172.21.15.6:0/2596175156 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f277406d660 msgr2=0x7f277406fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:09.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.942+0000 7f2792891700 1 --2- 172.21.15.6:0/2596175156 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f277406d660 0x7f277406fb20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f277c005fd0 tx=0x7f277c009310 comp rx=0 tx=0).stop 2024-03-24T12:10:09.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.942+0000 7f2792891700 1 -- 172.21.15.6:0/2596175156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f278c105b20 msgr2=0x7f278c0fd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:09.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.942+0000 7f2792891700 1 --2- 172.21.15.6:0/2596175156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f278c105b20 0x7f278c0fd030 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2784007ee0 tx=0x7f278400d3b0 comp rx=0 tx=0).stop 2024-03-24T12:10:09.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.942+0000 7f2790e2e700 1 -- 172.21.15.6:0/2596175156 reap_dead start 2024-03-24T12:10:09.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.942+0000 7f2792891700 1 -- 172.21.15.6:0/2596175156 shutdown_connections 2024-03-24T12:10:09.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.942+0000 7f2792891700 1 -- 172.21.15.6:0/2596175156 >> 172.21.15.6:0/2596175156 conn(0x7f278c077140 msgr2=0x7f278c1006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:09.942 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.942+0000 7f2792891700 1 -- 172.21.15.6:0/2596175156 shutdown_connections 2024-03-24T12:10:09.943 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.942+0000 7f2792891700 1 -- 172.21.15.6:0/2596175156 wait complete. 2024-03-24T12:10:10.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.079+0000 7fdf54125700 1 -- 172.21.15.6:0/2962638206 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf4c106ce0 msgr2=0x7fdf4c1070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:10.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.079+0000 7fdf54125700 1 --2- 172.21.15.6:0/2962638206 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf4c106ce0 0x7fdf4c1070c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009990 tx=0x7fdf3c009ca0 comp rx=0 tx=0).stop 2024-03-24T12:10:10.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.079+0000 7fdf54125700 1 -- 172.21.15.6:0/2962638206 shutdown_connections 2024-03-24T12:10:10.031 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.079+0000 7fdf54125700 1 --2- 172.21.15.6:0/2962638206 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf4c100c20 0x7fdf4c10eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:10.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.079+0000 7fdf54125700 1 --2- 172.21.15.6:0/2962638206 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf4c106ce0 0x7fdf4c1070c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:10.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.079+0000 7fdf54125700 1 --2- 172.21.15.6:0/2962638206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf4c100260 0x7fdf4c1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:10.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.079+0000 7fdf54125700 1 -- 172.21.15.6:0/2962638206 >> 172.21.15.6:0/2962638206 conn(0x7fdf4c077170 msgr2=0x7fdf4c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:10.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.079+0000 7fdf54125700 1 -- 172.21.15.6:0/2962638206 shutdown_connections 2024-03-24T12:10:10.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.080+0000 7fdf54125700 1 -- 172.21.15.6:0/2962638206 wait complete. 2024-03-24T12:10:10.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.080+0000 7fdf54125700 1 Processor -- start 2024-03-24T12:10:10.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.080+0000 7fdf54125700 1 -- start start 2024-03-24T12:10:10.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.080+0000 7fdf54125700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf4c100260 0x7fdf4c101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:10.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.080+0000 7fdf54125700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf4c100c20 0x7fdf4c101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:10.032 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.080+0000 7fdf54125700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf4c106ce0 0x7fdf4c101ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:10.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.080+0000 7fdf54125700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c1a5b70 con 0x7fdf4c100c20 2024-03-24T12:10:10.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.080+0000 7fdf54125700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c1a5ce0 con 0x7fdf4c106ce0 2024-03-24T12:10:10.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.080+0000 7fdf54125700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c1a5ea0 con 0x7fdf4c100260 2024-03-24T12:10:10.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.080+0000 7fdf51ec1700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf4c100260 0x7fdf4c101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:10.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.080+0000 7fdf516c0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf4c100c20 0x7fdf4c101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:10.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.080+0000 7fdf51ec1700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf4c100260 0x7fdf4c101440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:35298/0 (socket says 172.21.15.6:35298) 2024-03-24T12:10:10.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.080+0000 7fdf51ec1700 1 -- 172.21.15.6:0/2511533537 learned_addr learned my addr 172.21.15.6:0/2511533537 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:10:10.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.081+0000 7fdf526c2700 1 --2- 172.21.15.6:0/2511533537 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf4c106ce0 0x7fdf4c101ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:10.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.081+0000 7fdf516c0700 1 -- 172.21.15.6:0/2511533537 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf4c100260 msgr2=0x7fdf4c101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:10.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.081+0000 7fdf516c0700 1 --2- 172.21.15.6:0/2511533537 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf4c100260 0x7fdf4c101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:10.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.081+0000 7fdf516c0700 1 -- 172.21.15.6:0/2511533537 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf4c106ce0 msgr2=0x7fdf4c101ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:10.033 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.081+0000 7fdf516c0700 1 --2- 172.21.15.6:0/2511533537 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf4c106ce0 0x7fdf4c101ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:10.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.081+0000 7fdf516c0700 1 -- 172.21.15.6:0/2511533537 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf44009c40 con 0x7fdf4c100c20 2024-03-24T12:10:10.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.081+0000 7fdf51ec1700 1 --2- 172.21.15.6:0/2511533537 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf4c100260 0x7fdf4c101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:10:10.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.081+0000 7fdf516c0700 1 --2- 172.21.15.6:0/2511533537 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf4c100c20 0x7fdf4c101980 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800c970 tx=0x7fdf4800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:10.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.081+0000 7fdf42ffd700 1 -- 172.21.15.6:0/2511533537 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf48004440 con 0x7fdf4c100c20 2024-03-24T12:10:10.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.081+0000 7fdf42ffd700 1 -- 172.21.15.6:0/2511533537 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fdf48003b00 con 0x7fdf4c100c20 2024-03-24T12:10:10.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.081+0000 7fdf54125700 1 -- 172.21.15.6:0/2511533537 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf3c0095f0 con 0x7fdf4c100c20 2024-03-24T12:10:10.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.081+0000 7fdf42ffd700 1 -- 172.21.15.6:0/2511533537 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf48007a70 con 0x7fdf4c100c20 2024-03-24T12:10:10.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.081+0000 7fdf54125700 1 -- 172.21.15.6:0/2511533537 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4c1a6540 con 0x7fdf4c100c20 2024-03-24T12:10:10.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.082+0000 7fdf54125700 1 -- 172.21.15.6:0/2511533537 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdf4c111630 con 0x7fdf4c100c20 2024-03-24T12:10:10.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.083+0000 7fdf42ffd700 1 -- 172.21.15.6:0/2511533537 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7fdf48007bd0 con 0x7fdf4c100c20 2024-03-24T12:10:10.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.083+0000 7fdf42ffd700 1 --2- 172.21.15.6:0/2511533537 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fdf3806d660 0x7fdf3806fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:10.034 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.083+0000 7fdf42ffd700 1 -- 172.21.15.6:0/2511533537 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7fdf4808d1a0 con 0x7fdf4c100c20 2024-03-24T12:10:10.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.083+0000 7fdf51ec1700 1 --2- 172.21.15.6:0/2511533537 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fdf3806d660 0x7fdf3806fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:10.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.083+0000 7fdf42ffd700 1 --2- 172.21.15.6:0/2511533537 >> [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601] conn(0x7fdf38077570 0x7fdf38079a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:10.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.083+0000 7fdf42ffd700 1 -- 172.21.15.6:0/2511533537 --> [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf3807a0e0 con 0x7fdf38077570 2024-03-24T12:10:10.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.083+0000 7fdf42ffd700 1 -- 172.21.15.6:0/2511533537 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf4808f860 con 0x7fdf4c100c20 2024-03-24T12:10:10.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.083+0000 7fdf526c2700 1 --2- 172.21.15.6:0/2511533537 >> [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601] conn(0x7fdf38077570 0x7fdf38079a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:10.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.084+0000 7fdf51ec1700 1 --2- 172.21.15.6:0/2511533537 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fdf3806d660 0x7fdf3806fb20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c006010 tx=0x7fdf3c00b280 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:10.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.084+0000 7fdf526c2700 1 --2- 172.21.15.6:0/2511533537 >> [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601] conn(0x7fdf38077570 0x7fdf38079a10 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:10.035 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.084+0000 7fdf42ffd700 1 -- 172.21.15.6:0/2511533537 <== osd.1 v2:172.21.15.6:6810/3140266601 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fdf3807a0e0 con 0x7fdf38077570 2024-03-24T12:10:10.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.101+0000 7fdf54125700 1 -- 172.21.15.6:0/2511533537 --> [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdf4c105100 con 0x7fdf38077570 2024-03-24T12:10:10.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.101+0000 7fdf42ffd700 1 -- 172.21.15.6:0/2511533537 <== osd.1 v2:172.21.15.6:6810/3140266601 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdf4c105100 con 0x7fdf38077570 2024-03-24T12:10:10.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.101+0000 7fdf54125700 1 -- 172.21.15.6:0/2511533537 >> [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601] conn(0x7fdf38077570 msgr2=0x7fdf38079a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:10.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.101+0000 7fdf54125700 1 --2- 172.21.15.6:0/2511533537 >> [v2:172.21.15.6:6810/3140266601,v1:172.21.15.6:6811/3140266601] conn(0x7fdf38077570 0x7fdf38079a10 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:10.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.101+0000 7fdf54125700 1 -- 172.21.15.6:0/2511533537 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fdf3806d660 msgr2=0x7fdf3806fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:10.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.101+0000 7fdf54125700 1 --2- 172.21.15.6:0/2511533537 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fdf3806d660 0x7fdf3806fb20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c006010 tx=0x7fdf3c00b280 comp rx=0 tx=0).stop 2024-03-24T12:10:10.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.101+0000 7fdf54125700 1 -- 172.21.15.6:0/2511533537 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf4c100c20 msgr2=0x7fdf4c101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:10.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.101+0000 7fdf54125700 1 --2- 172.21.15.6:0/2511533537 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf4c100c20 0x7fdf4c101980 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800c970 tx=0x7fdf4800cc80 comp rx=0 tx=0).stop 2024-03-24T12:10:10.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.101+0000 7fdf526c2700 1 -- 172.21.15.6:0/2511533537 reap_dead start 2024-03-24T12:10:10.036 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.102+0000 7fdf54125700 1 -- 172.21.15.6:0/2511533537 shutdown_connections 2024-03-24T12:10:10.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.102+0000 7fdf54125700 1 -- 172.21.15.6:0/2511533537 >> 172.21.15.6:0/2511533537 conn(0x7fdf4c077170 msgr2=0x7fdf4c1047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:10.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.102+0000 7fdf54125700 1 -- 172.21.15.6:0/2511533537 shutdown_connections 2024-03-24T12:10:10.037 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:09.102+0000 7fdf54125700 1 -- 172.21.15.6:0/2511533537 wait complete. 2024-03-24T12:10:10.593 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:10 smithi006 ceph-mon[30108]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-24T12:10:10.594 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:10 smithi006 ceph-mon[41734]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-24T12:10:10.620 INFO:teuthology.orchestra.run.smithi006.stdout:55834574893 2024-03-24T12:10:10.620 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-24T12:10:10.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:10 smithi090 ceph-mon[33646]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-24T12:10:11.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.179+0000 7fb7870c5700 1 -- 172.21.15.6:0/1041672973 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb780106bd0 msgr2=0x7fb780106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:11.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.179+0000 7fb7870c5700 1 --2- 172.21.15.6:0/1041672973 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb780106bd0 0x7fb780106fb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb7740098f0 tx=0x7fb774009c00 comp rx=0 tx=0).stop 2024-03-24T12:10:11.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.179+0000 7fb7870c5700 1 -- 172.21.15.6:0/1041672973 shutdown_connections 2024-03-24T12:10:11.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.179+0000 7fb7870c5700 1 --2- 172.21.15.6:0/1041672973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb780101560 0x7fb78010d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:11.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.179+0000 7fb7870c5700 1 --2- 172.21.15.6:0/1041672973 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb780106bd0 0x7fb780106fb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:11.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.179+0000 7fb7870c5700 1 --2- 172.21.15.6:0/1041672973 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb780100ba0 0x7fb780101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:11.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.179+0000 7fb7870c5700 1 -- 172.21.15.6:0/1041672973 >> 172.21.15.6:0/1041672973 conn(0x7fb780076bc0 msgr2=0x7fb780076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:11.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.179+0000 7fb7870c5700 1 -- 172.21.15.6:0/1041672973 shutdown_connections 2024-03-24T12:10:11.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.179+0000 7fb7870c5700 1 -- 172.21.15.6:0/1041672973 wait complete. 2024-03-24T12:10:11.181 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.179+0000 7fb7870c5700 1 Processor -- start 2024-03-24T12:10:11.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.180+0000 7fb7870c5700 1 -- start start 2024-03-24T12:10:11.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.180+0000 7fb7870c5700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb780100ba0 0x7fb78019d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:11.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.180+0000 7fb7870c5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb780101560 0x7fb78019d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:11.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.180+0000 7fb7870c5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb780106bd0 0x7fb780197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:11.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.180+0000 7fb7870c5700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb780197c60 con 0x7fb780101560 2024-03-24T12:10:11.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.180+0000 7fb7870c5700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb780197da0 con 0x7fb780106bd0 2024-03-24T12:10:11.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.180+0000 7fb7870c5700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb780197f60 con 0x7fb780100ba0 2024-03-24T12:10:11.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.180+0000 7fb784e61700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb780100ba0 0x7fb78019d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:11.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.180+0000 7fb77ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb780101560 0x7fb78019d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:11.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.180+0000 7fb784e61700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb780100ba0 0x7fb78019d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:35346/0 (socket says 172.21.15.6:35346) 2024-03-24T12:10:11.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.180+0000 7fb77ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb780101560 0x7fb78019d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49770/0 (socket says 172.21.15.6:49770) 2024-03-24T12:10:11.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.180+0000 7fb784e61700 1 -- 172.21.15.6:0/2257957331 learned_addr learned my addr 172.21.15.6:0/2257957331 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:10:11.182 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.180+0000 7fb785662700 1 --2- 172.21.15.6:0/2257957331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb780106bd0 0x7fb780197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:11.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.180+0000 7fb784e61700 1 -- 172.21.15.6:0/2257957331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb780106bd0 msgr2=0x7fb780197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:11.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.180+0000 7fb784e61700 1 --2- 172.21.15.6:0/2257957331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb780106bd0 0x7fb780197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:11.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.180+0000 7fb784e61700 1 -- 172.21.15.6:0/2257957331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb780101560 msgr2=0x7fb78019d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:11.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.180+0000 7fb784e61700 1 --2- 172.21.15.6:0/2257957331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb780101560 0x7fb78019d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:11.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.180+0000 7fb784e61700 1 -- 172.21.15.6:0/2257957331 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76c0095f0 con 0x7fb780100ba0 2024-03-24T12:10:11.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.180+0000 7fb77ffff700 1 --2- 172.21.15.6:0/2257957331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb780101560 0x7fb78019d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:10:11.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.180+0000 7fb784e61700 1 --2- 172.21.15.6:0/2257957331 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb780100ba0 0x7fb78019d3f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb774005950 tx=0x7fb77400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:11.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.180+0000 7fb77dffb700 1 -- 172.21.15.6:0/2257957331 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77401c070 con 0x7fb780100ba0 2024-03-24T12:10:11.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.180+0000 7fb77dffb700 1 -- 172.21.15.6:0/2257957331 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fb77400bae0 con 0x7fb780100ba0 2024-03-24T12:10:11.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.180+0000 7fb7870c5700 1 -- 172.21.15.6:0/2257957331 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb774009520 con 0x7fb780100ba0 2024-03-24T12:10:11.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.181+0000 7fb77dffb700 1 -- 172.21.15.6:0/2257957331 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7740038e0 con 0x7fb780100ba0 2024-03-24T12:10:11.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.181+0000 7fb7870c5700 1 -- 172.21.15.6:0/2257957331 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7801a42c0 con 0x7fb780100ba0 2024-03-24T12:10:11.183 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.181+0000 7fb7870c5700 1 -- 172.21.15.6:0/2257957331 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb7801103e0 con 0x7fb780100ba0 2024-03-24T12:10:11.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.182+0000 7fb77dffb700 1 -- 172.21.15.6:0/2257957331 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7fb774020030 con 0x7fb780100ba0 2024-03-24T12:10:11.184 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.183+0000 7fb77dffb700 1 --2- 172.21.15.6:0/2257957331 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fb77006d660 0x7fb77006fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:11.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.183+0000 7fb77dffb700 1 -- 172.21.15.6:0/2257957331 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7fb77408e1f0 con 0x7fb780100ba0 2024-03-24T12:10:11.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.183+0000 7fb77ffff700 1 --2- 172.21.15.6:0/2257957331 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fb77006d660 0x7fb77006fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:11.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.183+0000 7fb77dffb700 1 --2- 172.21.15.6:0/2257957331 >> [v2:172.21.15.90:6808/3585923902,v1:172.21.15.90:6809/3585923902] conn(0x7fb770077570 0x7fb770079a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:11.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.183+0000 7fb77dffb700 1 -- 172.21.15.6:0/2257957331 --> [v2:172.21.15.90:6808/3585923902,v1:172.21.15.90:6809/3585923902] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb77007a0e0 con 0x7fb770077570 2024-03-24T12:10:11.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.183+0000 7fb77dffb700 1 -- 172.21.15.6:0/2257957331 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7740908b0 con 0x7fb780100ba0 2024-03-24T12:10:11.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.183+0000 7fb77ffff700 1 --2- 172.21.15.6:0/2257957331 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fb77006d660 0x7fb77006fb20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb7800772a0 tx=0x7fb76c009310 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:11.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.183+0000 7fb785662700 1 --2- 172.21.15.6:0/2257957331 >> [v2:172.21.15.90:6808/3585923902,v1:172.21.15.90:6809/3585923902] conn(0x7fb770077570 0x7fb770079a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:11.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.183+0000 7fb785662700 1 --2- 172.21.15.6:0/2257957331 >> [v2:172.21.15.90:6808/3585923902,v1:172.21.15.90:6809/3585923902] conn(0x7fb770077570 0x7fb770079a10 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:11.185 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.184+0000 7fb77dffb700 1 -- 172.21.15.6:0/2257957331 <== osd.5 v2:172.21.15.90:6808/3585923902 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb77007a0e0 con 0x7fb770077570 2024-03-24T12:10:11.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.204+0000 7fb7870c5700 1 -- 172.21.15.6:0/2257957331 --> [v2:172.21.15.90:6808/3585923902,v1:172.21.15.90:6809/3585923902] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb78006ab10 con 0x7fb770077570 2024-03-24T12:10:11.203 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.204+0000 7fb77dffb700 1 -- 172.21.15.6:0/2257957331 <== osd.5 v2:172.21.15.90:6808/3585923902 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb78006ab10 con 0x7fb770077570 2024-03-24T12:10:11.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.205+0000 7fb7870c5700 1 -- 172.21.15.6:0/2257957331 >> [v2:172.21.15.90:6808/3585923902,v1:172.21.15.90:6809/3585923902] conn(0x7fb770077570 msgr2=0x7fb770079a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:11.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.205+0000 7fb7870c5700 1 --2- 172.21.15.6:0/2257957331 >> [v2:172.21.15.90:6808/3585923902,v1:172.21.15.90:6809/3585923902] conn(0x7fb770077570 0x7fb770079a10 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:11.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.205+0000 7fb7870c5700 1 -- 172.21.15.6:0/2257957331 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fb77006d660 msgr2=0x7fb77006fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:11.204 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.205+0000 7fb7870c5700 1 --2- 172.21.15.6:0/2257957331 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fb77006d660 0x7fb77006fb20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb7800772a0 tx=0x7fb76c009310 comp rx=0 tx=0).stop 2024-03-24T12:10:11.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.205+0000 7fb7870c5700 1 -- 172.21.15.6:0/2257957331 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb780100ba0 msgr2=0x7fb78019d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:11.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.205+0000 7fb7870c5700 1 --2- 172.21.15.6:0/2257957331 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb780100ba0 0x7fb78019d3f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb774005950 tx=0x7fb77400fa00 comp rx=0 tx=0).stop 2024-03-24T12:10:11.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.205+0000 7fb785662700 1 -- 172.21.15.6:0/2257957331 reap_dead start 2024-03-24T12:10:11.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.205+0000 7fb7870c5700 1 -- 172.21.15.6:0/2257957331 shutdown_connections 2024-03-24T12:10:11.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.205+0000 7fb7870c5700 1 -- 172.21.15.6:0/2257957331 >> 172.21.15.6:0/2257957331 conn(0x7fb780076bc0 msgr2=0x7fb78006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:11.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.205+0000 7fb7870c5700 1 -- 172.21.15.6:0/2257957331 shutdown_connections 2024-03-24T12:10:11.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:11.205+0000 7fb7870c5700 1 -- 172.21.15.6:0/2257957331 wait complete. 2024-03-24T12:10:12.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:10:12 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:10:12.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:12 smithi006 ceph-mon[30108]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-24T12:10:12.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:12 smithi006 ceph-mon[41734]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-24T12:10:12.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:12 smithi090 ceph-mon[33646]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-24T12:10:12.975 INFO:teuthology.orchestra.run.smithi006.stdout:197568495635 2024-03-24T12:10:12.976 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-24T12:10:13.133 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.134+0000 7f50b20a9700 1 -- 172.21.15.6:0/1217458146 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f50ac106ce0 msgr2=0x7f50ac1070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:13.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.134+0000 7f50b20a9700 1 --2- 172.21.15.6:0/1217458146 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f50ac106ce0 0x7f50ac1070c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f50a00098f0 tx=0x7f50a0009c00 comp rx=0 tx=0).stop 2024-03-24T12:10:13.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.134+0000 7f50b20a9700 1 -- 172.21.15.6:0/1217458146 shutdown_connections 2024-03-24T12:10:13.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.134+0000 7f50b20a9700 1 --2- 172.21.15.6:0/1217458146 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50ac100c20 0x7f50ac10eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:13.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.134+0000 7f50b20a9700 1 --2- 172.21.15.6:0/1217458146 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f50ac106ce0 0x7f50ac1070c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:13.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.134+0000 7f50b20a9700 1 --2- 172.21.15.6:0/1217458146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50ac100260 0x7f50ac1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:13.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.134+0000 7f50b20a9700 1 -- 172.21.15.6:0/1217458146 >> 172.21.15.6:0/1217458146 conn(0x7f50ac077170 msgr2=0x7f50ac0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:13.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.134+0000 7f50b20a9700 1 -- 172.21.15.6:0/1217458146 shutdown_connections 2024-03-24T12:10:13.134 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.134+0000 7f50b20a9700 1 -- 172.21.15.6:0/1217458146 wait complete. 2024-03-24T12:10:13.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.135+0000 7f50b20a9700 1 Processor -- start 2024-03-24T12:10:13.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.135+0000 7f50b20a9700 1 -- start start 2024-03-24T12:10:13.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.135+0000 7f50b20a9700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f50ac100260 0x7f50ac19d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:13.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.135+0000 7f50b20a9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50ac100c20 0x7f50ac19db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:13.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.135+0000 7f50b20a9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50ac106ce0 0x7f50ac1976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:13.136 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.135+0000 7f50b20a9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac197e70 con 0x7f50ac106ce0 2024-03-24T12:10:13.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.135+0000 7f50b20a9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac197fe0 con 0x7f50ac100c20 2024-03-24T12:10:13.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.135+0000 7f50b20a9700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac1981a0 con 0x7f50ac100260 2024-03-24T12:10:13.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.135+0000 7f50ab7fe700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f50ac100260 0x7f50ac19d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:13.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.135+0000 7f50ab7fe700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f50ac100260 0x7f50ac19d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:35370/0 (socket says 172.21.15.6:35370) 2024-03-24T12:10:13.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.135+0000 7f50ab7fe700 1 -- 172.21.15.6:0/1909695345 learned_addr learned my addr 172.21.15.6:0/1909695345 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:10:13.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.135+0000 7f50aaffd700 1 --2- 172.21.15.6:0/1909695345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50ac100c20 0x7f50ac19db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:13.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.136+0000 7f50abfff700 1 --2- 172.21.15.6:0/1909695345 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50ac106ce0 0x7f50ac1976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:13.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.136+0000 7f50ab7fe700 1 -- 172.21.15.6:0/1909695345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50ac100c20 msgr2=0x7f50ac19db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:13.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.136+0000 7f50ab7fe700 1 --2- 172.21.15.6:0/1909695345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50ac100c20 0x7f50ac19db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:13.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.136+0000 7f50ab7fe700 1 -- 172.21.15.6:0/1909695345 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50ac106ce0 msgr2=0x7f50ac1976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:13.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.136+0000 7f50ab7fe700 1 --2- 172.21.15.6:0/1909695345 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50ac106ce0 0x7f50ac1976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:13.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.136+0000 7f50ab7fe700 1 -- 172.21.15.6:0/1909695345 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c0095f0 con 0x7f50ac100260 2024-03-24T12:10:13.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.136+0000 7f50abfff700 1 --2- 172.21.15.6:0/1909695345 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f50ac106ce0 0x7f50ac1976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T12:10:13.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.136+0000 7f50ab7fe700 1 --2- 172.21.15.6:0/1909695345 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f50ac100260 0x7f50ac19d5f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f50a0011040 tx=0x7f50a000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:13.137 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.136+0000 7f50a8ff9700 1 -- 172.21.15.6:0/1909695345 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a001c070 con 0x7f50ac100260 2024-03-24T12:10:13.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.136+0000 7f50a8ff9700 1 -- 172.21.15.6:0/1909695345 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f50a000bae0 con 0x7f50ac100260 2024-03-24T12:10:13.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.136+0000 7f50b20a9700 1 -- 172.21.15.6:0/1909695345 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a0009520 con 0x7f50ac100260 2024-03-24T12:10:13.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.136+0000 7f50a8ff9700 1 -- 172.21.15.6:0/1909695345 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a0004930 con 0x7f50ac100260 2024-03-24T12:10:13.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.136+0000 7f50b20a9700 1 -- 172.21.15.6:0/1909695345 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ac1a43a0 con 0x7f50ac100260 2024-03-24T12:10:13.138 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.137+0000 7f50b20a9700 1 -- 172.21.15.6:0/1909695345 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f50ac1115a0 con 0x7f50ac100260 2024-03-24T12:10:13.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.138+0000 7f50a8ff9700 1 -- 172.21.15.6:0/1909695345 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7f50a0020070 con 0x7f50ac100260 2024-03-24T12:10:13.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.138+0000 7f50a8ff9700 1 --2- 172.21.15.6:0/1909695345 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f509406d660 0x7f509406fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:13.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.138+0000 7f50aaffd700 1 --2- 172.21.15.6:0/1909695345 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f509406d660 0x7f509406fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:13.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.138+0000 7f50a8ff9700 1 -- 172.21.15.6:0/1909695345 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f50a008e120 con 0x7f50ac100260 2024-03-24T12:10:13.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.138+0000 7f50a8ff9700 1 --2- 172.21.15.6:0/1909695345 >> [v2:172.21.15.90:6824/1629523727,v1:172.21.15.90:6825/1629523727] conn(0x7f50940775d0 0x7f5094079a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:13.139 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.138+0000 7f50a8ff9700 1 -- 172.21.15.6:0/1909695345 --> [v2:172.21.15.90:6824/1629523727,v1:172.21.15.90:6825/1629523727] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f509407a140 con 0x7f50940775d0 2024-03-24T12:10:13.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.138+0000 7f50a8ff9700 1 -- 172.21.15.6:0/1909695345 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50a00907e0 con 0x7f50ac100260 2024-03-24T12:10:13.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.139+0000 7f50abfff700 1 --2- 172.21.15.6:0/1909695345 >> [v2:172.21.15.90:6824/1629523727,v1:172.21.15.90:6825/1629523727] conn(0x7f50940775d0 0x7f5094079a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:13.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.139+0000 7f50aaffd700 1 --2- 172.21.15.6:0/1909695345 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f509406d660 0x7f509406fb20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f50ac198e80 tx=0x7f509c009310 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:13.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.139+0000 7f50abfff700 1 --2- 172.21.15.6:0/1909695345 >> [v2:172.21.15.90:6824/1629523727,v1:172.21.15.90:6825/1629523727] conn(0x7f50940775d0 0x7f5094079a70 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:13.140 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.140+0000 7f50a8ff9700 1 -- 172.21.15.6:0/1909695345 <== osd.7 v2:172.21.15.90:6824/1629523727 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f509407a140 con 0x7f50940775d0 2024-03-24T12:10:13.157 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.158+0000 7f50b20a9700 1 -- 172.21.15.6:0/1909695345 --> [v2:172.21.15.90:6824/1629523727,v1:172.21.15.90:6825/1629523727] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f50ac105100 con 0x7f50940775d0 2024-03-24T12:10:13.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.158+0000 7f50a8ff9700 1 -- 172.21.15.6:0/1909695345 <== osd.7 v2:172.21.15.90:6824/1629523727 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f50ac105100 con 0x7f50940775d0 2024-03-24T12:10:13.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.159+0000 7f50b20a9700 1 -- 172.21.15.6:0/1909695345 >> [v2:172.21.15.90:6824/1629523727,v1:172.21.15.90:6825/1629523727] conn(0x7f50940775d0 msgr2=0x7f5094079a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:13.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.159+0000 7f50b20a9700 1 --2- 172.21.15.6:0/1909695345 >> [v2:172.21.15.90:6824/1629523727,v1:172.21.15.90:6825/1629523727] conn(0x7f50940775d0 0x7f5094079a70 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:13.158 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.159+0000 7f50b20a9700 1 -- 172.21.15.6:0/1909695345 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f509406d660 msgr2=0x7f509406fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:13.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.159+0000 7f50b20a9700 1 --2- 172.21.15.6:0/1909695345 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f509406d660 0x7f509406fb20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f50ac198e80 tx=0x7f509c009310 comp rx=0 tx=0).stop 2024-03-24T12:10:13.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.159+0000 7f50b20a9700 1 -- 172.21.15.6:0/1909695345 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f50ac100260 msgr2=0x7f50ac19d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:13.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.159+0000 7f50b20a9700 1 --2- 172.21.15.6:0/1909695345 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f50ac100260 0x7f50ac19d5f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f50a0011040 tx=0x7f50a000fa00 comp rx=0 tx=0).stop 2024-03-24T12:10:13.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.159+0000 7f50abfff700 1 -- 172.21.15.6:0/1909695345 reap_dead start 2024-03-24T12:10:13.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.159+0000 7f50b20a9700 1 -- 172.21.15.6:0/1909695345 shutdown_connections 2024-03-24T12:10:13.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.159+0000 7f50b20a9700 1 -- 172.21.15.6:0/1909695345 >> 172.21.15.6:0/1909695345 conn(0x7f50ac077170 msgr2=0x7f50ac1047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:13.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.159+0000 7f50b20a9700 1 -- 172.21.15.6:0/1909695345 shutdown_connections 2024-03-24T12:10:13.159 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:13.159+0000 7f50b20a9700 1 -- 172.21.15.6:0/1909695345 wait complete. 2024-03-24T12:10:13.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:13 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:10:13.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:13 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:10:13.785 INFO:teuthology.orchestra.run.smithi006.stdout:163208757272 2024-03-24T12:10:13.785 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-24T12:10:13.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:13 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:10:14.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.426+0000 7f87d6ed1700 1 -- 172.21.15.6:0/2889127971 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87d0105670 msgr2=0x7f87d010d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:14.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.426+0000 7f87d6ed1700 1 --2- 172.21.15.6:0/2889127971 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87d0105670 0x7f87d010d240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f87c800b0a0 tx=0x7f87c800b3b0 comp rx=0 tx=0).stop 2024-03-24T12:10:14.427 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.426+0000 7f87d6ed1700 1 -- 172.21.15.6:0/2889127971 shutdown_connections 2024-03-24T12:10:14.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.426+0000 7f87d6ed1700 1 --2- 172.21.15.6:0/2889127971 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87d0105670 0x7f87d010d240 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:14.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.426+0000 7f87d6ed1700 1 --2- 172.21.15.6:0/2889127971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87d0100590 0x7f87d0100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:14.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.426+0000 7f87d6ed1700 1 --2- 172.21.15.6:0/2889127971 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f87d0100f40 0x7f87d0104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:14.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.426+0000 7f87d6ed1700 1 -- 172.21.15.6:0/2889127971 >> 172.21.15.6:0/2889127971 conn(0x7f87d0077170 msgr2=0x7f87d00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:14.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.426+0000 7f87d6ed1700 1 -- 172.21.15.6:0/2889127971 shutdown_connections 2024-03-24T12:10:14.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.426+0000 7f87d6ed1700 1 -- 172.21.15.6:0/2889127971 wait complete. 2024-03-24T12:10:14.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.427+0000 7f87d6ed1700 1 Processor -- start 2024-03-24T12:10:14.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.427+0000 7f87d6ed1700 1 -- start start 2024-03-24T12:10:14.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.428+0000 7f87d6ed1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87d0100590 0x7f87d019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:14.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.428+0000 7f87d6ed1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87d0100f40 0x7f87d019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:14.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.428+0000 7f87d6ed1700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f87d0105670 0x7f87d0197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:14.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.428+0000 7f87d6ed1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d0197ec0 con 0x7f87d0100590 2024-03-24T12:10:14.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.428+0000 7f87d6ed1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d0198030 con 0x7f87d0100f40 2024-03-24T12:10:14.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.428+0000 7f87d6ed1700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d01981f0 con 0x7f87d0105670 2024-03-24T12:10:14.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.428+0000 7f87d4c6d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87d0100590 0x7f87d019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:14.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.428+0000 7f87d4c6d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87d0100590 0x7f87d019d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:49794/0 (socket says 172.21.15.6:49794) 2024-03-24T12:10:14.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.428+0000 7f87d4c6d700 1 -- 172.21.15.6:0/1839331950 learned_addr learned my addr 172.21.15.6:0/1839331950 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:10:14.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.428+0000 7f87cffff700 1 --2- 172.21.15.6:0/1839331950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87d0100f40 0x7f87d019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:14.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.428+0000 7f87d546e700 1 --2- 172.21.15.6:0/1839331950 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f87d0105670 0x7f87d0197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:14.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.428+0000 7f87d4c6d700 1 -- 172.21.15.6:0/1839331950 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f87d0105670 msgr2=0x7f87d0197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:14.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.428+0000 7f87d4c6d700 1 --2- 172.21.15.6:0/1839331950 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f87d0105670 0x7f87d0197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:14.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.428+0000 7f87d4c6d700 1 -- 172.21.15.6:0/1839331950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87d0100f40 msgr2=0x7f87d019db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:14.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.428+0000 7f87d4c6d700 1 --2- 172.21.15.6:0/1839331950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87d0100f40 0x7f87d019db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:14.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.428+0000 7f87d4c6d700 1 -- 172.21.15.6:0/1839331950 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c8009d00 con 0x7f87d0100590 2024-03-24T12:10:14.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.429+0000 7f87d4c6d700 1 --2- 172.21.15.6:0/1839331950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87d0100590 0x7f87d019d640 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f87c400e9a0 tx=0x7f87c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:14.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.429+0000 7f87cdffb700 1 -- 172.21.15.6:0/1839331950 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c4009790 con 0x7f87d0100590 2024-03-24T12:10:14.430 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.429+0000 7f87d6ed1700 1 -- 172.21.15.6:0/1839331950 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d01984d0 con 0x7f87d0100590 2024-03-24T12:10:14.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.429+0000 7f87cdffb700 1 -- 172.21.15.6:0/1839331950 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f87c400ce90 con 0x7f87d0100590 2024-03-24T12:10:14.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.429+0000 7f87cdffb700 1 -- 172.21.15.6:0/1839331950 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c4018420 con 0x7f87d0100590 2024-03-24T12:10:14.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.430+0000 7f87d6ed1700 1 -- 172.21.15.6:0/1839331950 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d01a4610 con 0x7f87d0100590 2024-03-24T12:10:14.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.430+0000 7f87d6ed1700 1 -- 172.21.15.6:0/1839331950 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f87d010f390 con 0x7f87d0100590 2024-03-24T12:10:14.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.430+0000 7f87cdffb700 1 -- 172.21.15.6:0/1839331950 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7f87c4018580 con 0x7f87d0100590 2024-03-24T12:10:14.431 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.431+0000 7f87cdffb700 1 --2- 172.21.15.6:0/1839331950 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f87c006d660 0x7f87c006fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:14.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.431+0000 7f87cdffb700 1 -- 172.21.15.6:0/1839331950 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f87c4014070 con 0x7f87d0100590 2024-03-24T12:10:14.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.431+0000 7f87cffff700 1 --2- 172.21.15.6:0/1839331950 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f87c006d660 0x7f87c006fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:14.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.431+0000 7f87cdffb700 1 --2- 172.21.15.6:0/1839331950 >> [v2:172.21.15.6:6802/195441355,v1:172.21.15.6:6803/195441355] conn(0x7f87c0077470 0x7f87c0079910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:14.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.431+0000 7f87cdffb700 1 -- 172.21.15.6:0/1839331950 --> [v2:172.21.15.6:6802/195441355,v1:172.21.15.6:6803/195441355] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f87c0079fe0 con 0x7f87c0077470 2024-03-24T12:10:14.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.431+0000 7f87cdffb700 1 -- 172.21.15.6:0/1839331950 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87c4056960 con 0x7f87d0100590 2024-03-24T12:10:14.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.431+0000 7f87d546e700 1 --2- 172.21.15.6:0/1839331950 >> [v2:172.21.15.6:6802/195441355,v1:172.21.15.6:6803/195441355] conn(0x7f87c0077470 0x7f87c0079910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:14.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.432+0000 7f87cffff700 1 --2- 172.21.15.6:0/1839331950 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f87c006d660 0x7f87c006fb20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f87d0198ed0 tx=0x7f87bc010040 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:14.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.432+0000 7f87d546e700 1 --2- 172.21.15.6:0/1839331950 >> [v2:172.21.15.6:6802/195441355,v1:172.21.15.6:6803/195441355] conn(0x7f87c0077470 0x7f87c0079910 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:14.432 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.432+0000 7f87cdffb700 1 -- 172.21.15.6:0/1839331950 <== osd.0 v2:172.21.15.6:6802/195441355 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f87c0079fe0 con 0x7f87c0077470 2024-03-24T12:10:14.448 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.449+0000 7f87d6ed1700 1 -- 172.21.15.6:0/1839331950 --> [v2:172.21.15.6:6802/195441355,v1:172.21.15.6:6803/195441355] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f87d0062300 con 0x7f87c0077470 2024-03-24T12:10:14.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.449+0000 7f87cdffb700 1 -- 172.21.15.6:0/1839331950 <== osd.0 v2:172.21.15.6:6802/195441355 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f87d0062300 con 0x7f87c0077470 2024-03-24T12:10:14.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.449+0000 7f87d6ed1700 1 -- 172.21.15.6:0/1839331950 >> [v2:172.21.15.6:6802/195441355,v1:172.21.15.6:6803/195441355] conn(0x7f87c0077470 msgr2=0x7f87c0079910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:14.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.449+0000 7f87d6ed1700 1 --2- 172.21.15.6:0/1839331950 >> [v2:172.21.15.6:6802/195441355,v1:172.21.15.6:6803/195441355] conn(0x7f87c0077470 0x7f87c0079910 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:14.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.450+0000 7f87d6ed1700 1 -- 172.21.15.6:0/1839331950 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f87c006d660 msgr2=0x7f87c006fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:14.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.450+0000 7f87d6ed1700 1 --2- 172.21.15.6:0/1839331950 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f87c006d660 0x7f87c006fb20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f87d0198ed0 tx=0x7f87bc010040 comp rx=0 tx=0).stop 2024-03-24T12:10:14.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.450+0000 7f87d6ed1700 1 -- 172.21.15.6:0/1839331950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87d0100590 msgr2=0x7f87d019d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:14.449 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.450+0000 7f87d6ed1700 1 --2- 172.21.15.6:0/1839331950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87d0100590 0x7f87d019d640 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f87c400e9a0 tx=0x7f87c400ecb0 comp rx=0 tx=0).stop 2024-03-24T12:10:14.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.450+0000 7f87d546e700 1 -- 172.21.15.6:0/1839331950 reap_dead start 2024-03-24T12:10:14.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.450+0000 7f87d6ed1700 1 -- 172.21.15.6:0/1839331950 shutdown_connections 2024-03-24T12:10:14.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.450+0000 7f87d6ed1700 1 -- 172.21.15.6:0/1839331950 >> 172.21.15.6:0/1839331950 conn(0x7f87d0077170 msgr2=0x7f87d0103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:14.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.450+0000 7f87d6ed1700 1 -- 172.21.15.6:0/1839331950 shutdown_connections 2024-03-24T12:10:14.450 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.450+0000 7f87d6ed1700 1 -- 172.21.15.6:0/1839331950 wait complete. 2024-03-24T12:10:14.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.636+0000 7fbd72f9d700 1 -- 172.21.15.6:0/3466229547 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbd6c101530 msgr2=0x7fbd6c10d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:14.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.636+0000 7fbd72f9d700 1 --2- 172.21.15.6:0/3466229547 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbd6c101530 0x7fbd6c10d9c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400b0a0 tx=0x7fbd6400b3b0 comp rx=0 tx=0).stop 2024-03-24T12:10:14.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.636+0000 7fbd72f9d700 1 -- 172.21.15.6:0/3466229547 shutdown_connections 2024-03-24T12:10:14.636 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.636+0000 7fbd72f9d700 1 --2- 172.21.15.6:0/3466229547 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbd6c101530 0x7fbd6c10d9c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:14.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.636+0000 7fbd72f9d700 1 --2- 172.21.15.6:0/3466229547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd6c106b80 0x7fbd6c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:14.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.636+0000 7fbd72f9d700 1 --2- 172.21.15.6:0/3466229547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd6c100b70 0x7fbd6c100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:14.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.636+0000 7fbd72f9d700 1 -- 172.21.15.6:0/3466229547 >> 172.21.15.6:0/3466229547 conn(0x7fbd6c077aa0 msgr2=0x7fbd6c077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:14.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.637+0000 7fbd72f9d700 1 -- 172.21.15.6:0/3466229547 shutdown_connections 2024-03-24T12:10:14.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.637+0000 7fbd72f9d700 1 -- 172.21.15.6:0/3466229547 wait complete. 2024-03-24T12:10:14.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.637+0000 7fbd72f9d700 1 Processor -- start 2024-03-24T12:10:14.637 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.637+0000 7fbd72f9d700 1 -- start start 2024-03-24T12:10:14.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.637+0000 7fbd72f9d700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbd6c100b70 0x7fbd6c196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:14.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.637+0000 7fbd72f9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd6c101530 0x7fbd6c196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:14.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.637+0000 7fbd72f9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd6c106b80 0x7fbd6c19afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:14.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.637+0000 7fbd72f9d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c19b5f0 con 0x7fbd6c106b80 2024-03-24T12:10:14.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.637+0000 7fbd72f9d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c19b760 con 0x7fbd6c101530 2024-03-24T12:10:14.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.637+0000 7fbd72f9d700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c19b920 con 0x7fbd6c100b70 2024-03-24T12:10:14.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.637+0000 7fbd70d39700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbd6c100b70 0x7fbd6c196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:14.639 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.637+0000 7fbd70d39700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbd6c100b70 0x7fbd6c196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:35404/0 (socket says 172.21.15.6:35404) 2024-03-24T12:10:14.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.637+0000 7fbd70d39700 1 -- 172.21.15.6:0/3426557561 learned_addr learned my addr 172.21.15.6:0/3426557561 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:10:14.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.637+0000 7fbd7153a700 1 --2- 172.21.15.6:0/3426557561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd6c106b80 0x7fbd6c19afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:14.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.638+0000 7fbd6bfff700 1 --2- 172.21.15.6:0/3426557561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd6c101530 0x7fbd6c196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:14.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.638+0000 7fbd7153a700 1 -- 172.21.15.6:0/3426557561 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbd6c100b70 msgr2=0x7fbd6c196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:14.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.638+0000 7fbd7153a700 1 --2- 172.21.15.6:0/3426557561 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbd6c100b70 0x7fbd6c196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:14.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.638+0000 7fbd7153a700 1 -- 172.21.15.6:0/3426557561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd6c101530 msgr2=0x7fbd6c196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:14.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.638+0000 7fbd7153a700 1 --2- 172.21.15.6:0/3426557561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd6c101530 0x7fbd6c196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:14.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.638+0000 7fbd7153a700 1 -- 172.21.15.6:0/3426557561 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd64009d00 con 0x7fbd6c106b80 2024-03-24T12:10:14.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.638+0000 7fbd7153a700 1 --2- 172.21.15.6:0/3426557561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd6c106b80 0x7fbd6c19afe0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400b070 tx=0x7fbd640078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:14.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.638+0000 7fbd69ffb700 1 -- 172.21.15.6:0/3426557561 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6400e040 con 0x7fbd6c106b80 2024-03-24T12:10:14.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.638+0000 7fbd69ffb700 1 -- 172.21.15.6:0/3426557561 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fbd6401d470 con 0x7fbd6c106b80 2024-03-24T12:10:14.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.638+0000 7fbd72f9d700 1 -- 172.21.15.6:0/3426557561 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd6c19bba0 con 0x7fbd6c106b80 2024-03-24T12:10:14.640 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.638+0000 7fbd69ffb700 1 -- 172.21.15.6:0/3426557561 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd640052c0 con 0x7fbd6c106b80 2024-03-24T12:10:14.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.638+0000 7fbd72f9d700 1 -- 172.21.15.6:0/3426557561 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6c069b70 con 0x7fbd6c106b80 2024-03-24T12:10:14.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.639+0000 7fbd72f9d700 1 -- 172.21.15.6:0/3426557561 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbd6c19be80 con 0x7fbd6c106b80 2024-03-24T12:10:14.641 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.640+0000 7fbd69ffb700 1 -- 172.21.15.6:0/3426557561 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7fbd64004a50 con 0x7fbd6c106b80 2024-03-24T12:10:14.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.640+0000 7fbd69ffb700 1 --2- 172.21.15.6:0/3426557561 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fbd5c06d660 0x7fbd5c06fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:14.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.640+0000 7fbd69ffb700 1 -- 172.21.15.6:0/3426557561 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7fbd6408ec70 con 0x7fbd6c106b80 2024-03-24T12:10:14.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.640+0000 7fbd70d39700 1 --2- 172.21.15.6:0/3426557561 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fbd5c06d660 0x7fbd5c06fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:14.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.640+0000 7fbd69ffb700 1 --2- 172.21.15.6:0/3426557561 >> [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884] conn(0x7fbd5c0773c0 0x7fbd5c079860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:14.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.640+0000 7fbd69ffb700 1 -- 172.21.15.6:0/3426557561 --> [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbd5c079f30 con 0x7fbd5c0773c0 2024-03-24T12:10:14.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.641+0000 7fbd69ffb700 1 -- 172.21.15.6:0/3426557561 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd6408f0c0 con 0x7fbd6c106b80 2024-03-24T12:10:14.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.641+0000 7fbd6bfff700 1 --2- 172.21.15.6:0/3426557561 >> [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884] conn(0x7fbd5c0773c0 0x7fbd5c079860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:14.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.641+0000 7fbd70d39700 1 --2- 172.21.15.6:0/3426557561 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fbd5c06d660 0x7fbd5c06fb20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbd60009d20 tx=0x7fbd60009260 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:14.642 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.641+0000 7fbd6bfff700 1 --2- 172.21.15.6:0/3426557561 >> [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884] conn(0x7fbd5c0773c0 0x7fbd5c079860 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:14.643 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.642+0000 7fbd69ffb700 1 -- 172.21.15.6:0/3426557561 <== osd.3 v2:172.21.15.6:6826/1341863884 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbd5c079f30 con 0x7fbd5c0773c0 2024-03-24T12:10:14.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.658+0000 7fbd72f9d700 1 -- 172.21.15.6:0/3426557561 --> [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbd6c106060 con 0x7fbd5c0773c0 2024-03-24T12:10:14.657 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.658+0000 7fbd69ffb700 1 -- 172.21.15.6:0/3426557561 <== osd.3 v2:172.21.15.6:6826/1341863884 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbd6c106060 con 0x7fbd5c0773c0 2024-03-24T12:10:14.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.659+0000 7fbd72f9d700 1 -- 172.21.15.6:0/3426557561 >> [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884] conn(0x7fbd5c0773c0 msgr2=0x7fbd5c079860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:14.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.659+0000 7fbd72f9d700 1 --2- 172.21.15.6:0/3426557561 >> [v2:172.21.15.6:6826/1341863884,v1:172.21.15.6:6827/1341863884] conn(0x7fbd5c0773c0 0x7fbd5c079860 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:14.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.659+0000 7fbd72f9d700 1 -- 172.21.15.6:0/3426557561 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fbd5c06d660 msgr2=0x7fbd5c06fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:14.658 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.659+0000 7fbd72f9d700 1 --2- 172.21.15.6:0/3426557561 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fbd5c06d660 0x7fbd5c06fb20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbd60009d20 tx=0x7fbd60009260 comp rx=0 tx=0).stop 2024-03-24T12:10:14.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.659+0000 7fbd72f9d700 1 -- 172.21.15.6:0/3426557561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd6c106b80 msgr2=0x7fbd6c19afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:14.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.659+0000 7fbd72f9d700 1 --2- 172.21.15.6:0/3426557561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbd6c106b80 0x7fbd6c19afe0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400b070 tx=0x7fbd640078e0 comp rx=0 tx=0).stop 2024-03-24T12:10:14.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.659+0000 7fbd7153a700 1 -- 172.21.15.6:0/3426557561 reap_dead start 2024-03-24T12:10:14.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.659+0000 7fbd72f9d700 1 -- 172.21.15.6:0/3426557561 shutdown_connections 2024-03-24T12:10:14.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.659+0000 7fbd72f9d700 1 -- 172.21.15.6:0/3426557561 >> 172.21.15.6:0/3426557561 conn(0x7fbd6c077aa0 msgr2=0x7fbd6c1046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:14.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.659+0000 7fbd72f9d700 1 -- 172.21.15.6:0/3426557561 shutdown_connections 2024-03-24T12:10:14.659 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:14.659+0000 7fbd72f9d700 1 -- 172.21.15.6:0/3426557561 wait complete. 2024-03-24T12:10:14.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:14 smithi006 ceph-mon[30108]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-24T12:10:14.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:14 smithi006 ceph-mon[41734]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-24T12:10:14.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:14 smithi090 ceph-mon[33646]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-24T12:10:15.266 INFO:teuthology.orchestra.run.smithi006.stdout:34359738418 2024-03-24T12:10:15.266 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-24T12:10:15.520 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:10:15.521 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:10:15.525 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:10:15.526 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:10:15.526 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:10:15.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:10:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:10:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:10:16.176 INFO:teuthology.orchestra.run.smithi006.stdout:231928233999 2024-03-24T12:10:16.176 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-24T12:10:16.543 INFO:teuthology.orchestra.run.smithi006.stdout:107374182436 2024-03-24T12:10:16.543 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-24T12:10:16.794 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:16 smithi006 ceph-mon[30108]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T12:10:16.794 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:16 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:10:16.795 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:16 smithi006 ceph-mon[41734]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T12:10:16.795 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:16 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:10:16.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:16 smithi090 ceph-mon[33646]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T12:10:16.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:16 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:10:17.189 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:10:18.752 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:18 smithi006 ceph-mon[30108]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T12:10:18.752 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:18 smithi006 ceph-mon[41734]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T12:10:18.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:18 smithi090 ceph-mon[33646]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-24T12:10:20.620 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:20 smithi006 ceph-mon[30108]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:20.621 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:20 smithi006 ceph-mon[41734]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:20.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:20 smithi090 ceph-mon[33646]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:22.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:10:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:10:22.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:22 smithi006 ceph-mon[30108]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:22.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:22 smithi006 ceph-mon[41734]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:22.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:22 smithi090 ceph-mon[33646]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:23.116 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:10:23.647 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:23 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:10:23.648 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:23 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:10:23.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:23 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:10:24.202 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.203+0000 7fbb66502700 1 -- 172.21.15.6:0/473748045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb6006afa0 msgr2=0x7fbb601068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:24.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.203+0000 7fbb66502700 1 --2- 172.21.15.6:0/473748045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb6006afa0 0x7fbb601068a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbb540098f0 tx=0x7fbb54009c00 comp rx=0 tx=0).stop 2024-03-24T12:10:24.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.203+0000 7fbb66502700 1 -- 172.21.15.6:0/473748045 shutdown_connections 2024-03-24T12:10:24.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.203+0000 7fbb66502700 1 --2- 172.21.15.6:0/473748045 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbb60106fd0 0x7fbb6010eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:24.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.203+0000 7fbb66502700 1 --2- 172.21.15.6:0/473748045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb6006a5f0 0x7fbb6006a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:24.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.203+0000 7fbb66502700 1 --2- 172.21.15.6:0/473748045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb6006afa0 0x7fbb601068a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:24.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.203+0000 7fbb66502700 1 -- 172.21.15.6:0/473748045 >> 172.21.15.6:0/473748045 conn(0x7fbb60077b10 msgr2=0x7fbb60077f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:24.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.203+0000 7fbb66502700 1 -- 172.21.15.6:0/473748045 shutdown_connections 2024-03-24T12:10:24.205 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.203+0000 7fbb66502700 1 -- 172.21.15.6:0/473748045 wait complete. 2024-03-24T12:10:24.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.203+0000 7fbb66502700 1 Processor -- start 2024-03-24T12:10:24.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.204+0000 7fbb66502700 1 -- start start 2024-03-24T12:10:24.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.204+0000 7fbb66502700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbb6006a5f0 0x7fbb60100c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:24.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.204+0000 7fbb66502700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb6006afa0 0x7fbb60101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:24.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.204+0000 7fbb66502700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb60106fd0 0x7fbb601016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:24.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.204+0000 7fbb66502700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb601a5ce0 con 0x7fbb60106fd0 2024-03-24T12:10:24.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.204+0000 7fbb66502700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb601a5e50 con 0x7fbb6006afa0 2024-03-24T12:10:24.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.204+0000 7fbb66502700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb601a6010 con 0x7fbb6006a5f0 2024-03-24T12:10:24.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.204+0000 7fbb5ffff700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbb6006a5f0 0x7fbb60100c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:24.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.204+0000 7fbb64a9f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb60106fd0 0x7fbb601016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:24.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.204+0000 7fbb5ffff700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbb6006a5f0 0x7fbb60100c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:41178/0 (socket says 172.21.15.6:41178) 2024-03-24T12:10:24.208 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.204+0000 7fbb5ffff700 1 -- 172.21.15.6:0/1959242733 learned_addr learned my addr 172.21.15.6:0/1959242733 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:10:24.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.204+0000 7fbb5f7fe700 1 --2- 172.21.15.6:0/1959242733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb6006afa0 0x7fbb60101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:24.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.204+0000 7fbb5ffff700 1 -- 172.21.15.6:0/1959242733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb6006afa0 msgr2=0x7fbb60101160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:24.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.204+0000 7fbb5ffff700 1 --2- 172.21.15.6:0/1959242733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb6006afa0 0x7fbb60101160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:24.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.204+0000 7fbb5ffff700 1 -- 172.21.15.6:0/1959242733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb60106fd0 msgr2=0x7fbb601016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:24.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.204+0000 7fbb5ffff700 1 --2- 172.21.15.6:0/1959242733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb60106fd0 0x7fbb601016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:24.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.204+0000 7fbb5ffff700 1 -- 172.21.15.6:0/1959242733 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb58009d00 con 0x7fbb6006a5f0 2024-03-24T12:10:24.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.204+0000 7fbb5f7fe700 1 --2- 172.21.15.6:0/1959242733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb6006afa0 0x7fbb60101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T12:10:24.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.204+0000 7fbb64a9f700 1 --2- 172.21.15.6:0/1959242733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb60106fd0 0x7fbb601016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:10:24.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.205+0000 7fbb5ffff700 1 --2- 172.21.15.6:0/1959242733 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbb6006a5f0 0x7fbb60100c20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000b530 tx=0x7fbb5000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:24.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.205+0000 7fbb5d7fa700 1 -- 172.21.15.6:0/1959242733 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb50011830 con 0x7fbb6006a5f0 2024-03-24T12:10:24.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.205+0000 7fbb5d7fa700 1 -- 172.21.15.6:0/1959242733 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fbb50011e10 con 0x7fbb6006a5f0 2024-03-24T12:10:24.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.205+0000 7fbb66502700 1 -- 172.21.15.6:0/1959242733 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb54009520 con 0x7fbb6006a5f0 2024-03-24T12:10:24.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.205+0000 7fbb5d7fa700 1 -- 172.21.15.6:0/1959242733 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb50010e30 con 0x7fbb6006a5f0 2024-03-24T12:10:24.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.205+0000 7fbb66502700 1 -- 172.21.15.6:0/1959242733 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb601a6680 con 0x7fbb6006a5f0 2024-03-24T12:10:24.209 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.206+0000 7fbb66502700 1 -- 172.21.15.6:0/1959242733 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb60068930 con 0x7fbb6006a5f0 2024-03-24T12:10:24.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.206+0000 7fbb5d7fa700 1 -- 172.21.15.6:0/1959242733 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7fbb50011990 con 0x7fbb6006a5f0 2024-03-24T12:10:24.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.207+0000 7fbb5d7fa700 1 --2- 172.21.15.6:0/1959242733 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fbb4806d660 0x7fbb4806fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:24.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.207+0000 7fbb5d7fa700 1 -- 172.21.15.6:0/1959242733 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7fbb5008ce40 con 0x7fbb6006a5f0 2024-03-24T12:10:24.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.207+0000 7fbb5f7fe700 1 --2- 172.21.15.6:0/1959242733 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fbb4806d660 0x7fbb4806fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:24.210 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.207+0000 7fbb5f7fe700 1 --2- 172.21.15.6:0/1959242733 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fbb4806d660 0x7fbb4806fb20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbb601026b0 tx=0x7fbb54009f90 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:24.211 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.212+0000 7fbb5d7fa700 1 -- 172.21.15.6:0/1959242733 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbb50056510 con 0x7fbb6006a5f0 2024-03-24T12:10:24.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.419+0000 7fbb66502700 1 -- 172.21.15.6:0/1959242733 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fbb60111250 con 0x7fbb6006a5f0 2024-03-24T12:10:24.419 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.419+0000 7fbb5d7fa700 1 -- 172.21.15.6:0/1959242733 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbb50015090 con 0x7fbb6006a5f0 2024-03-24T12:10:24.419 INFO:teuthology.orchestra.run.smithi006.stdout:197568495638 2024-03-24T12:10:24.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.422+0000 7fbb66502700 1 -- 172.21.15.6:0/1959242733 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fbb4806d660 msgr2=0x7fbb4806fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:24.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.422+0000 7fbb66502700 1 --2- 172.21.15.6:0/1959242733 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fbb4806d660 0x7fbb4806fb20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbb601026b0 tx=0x7fbb54009f90 comp rx=0 tx=0).stop 2024-03-24T12:10:24.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.422+0000 7fbb66502700 1 -- 172.21.15.6:0/1959242733 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbb6006a5f0 msgr2=0x7fbb60100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:24.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.422+0000 7fbb66502700 1 --2- 172.21.15.6:0/1959242733 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbb6006a5f0 0x7fbb60100c20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000b530 tx=0x7fbb5000b840 comp rx=0 tx=0).stop 2024-03-24T12:10:24.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.422+0000 7fbb66502700 1 -- 172.21.15.6:0/1959242733 shutdown_connections 2024-03-24T12:10:24.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.422+0000 7fbb66502700 1 --2- 172.21.15.6:0/1959242733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb60106fd0 0x7fbb601016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:24.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.422+0000 7fbb66502700 1 --2- 172.21.15.6:0/1959242733 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fbb4806d660 0x7fbb4806fb20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:24.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.422+0000 7fbb66502700 1 --2- 172.21.15.6:0/1959242733 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbb6006a5f0 0x7fbb60100c20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:24.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.422+0000 7fbb66502700 1 --2- 172.21.15.6:0/1959242733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb6006afa0 0x7fbb60101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:24.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.422+0000 7fbb66502700 1 -- 172.21.15.6:0/1959242733 >> 172.21.15.6:0/1959242733 conn(0x7fbb60077b10 msgr2=0x7fbb600fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:24.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.422+0000 7fbb66502700 1 -- 172.21.15.6:0/1959242733 shutdown_connections 2024-03-24T12:10:24.422 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.422+0000 7fbb66502700 1 -- 172.21.15.6:0/1959242733 wait complete. 2024-03-24T12:10:24.530 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:24 smithi006 ceph-mon[30108]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:10:24.530 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1959242733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-24T12:10:24.530 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:24 smithi006 ceph-mon[41734]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:10:24.530 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1959242733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-24T12:10:24.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:24 smithi090 ceph-mon[33646]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:10:24.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1959242733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-24T12:10:24.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.445+0000 7fcdc15d9700 1 -- 172.21.15.6:0/3097938929 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdbc102860 msgr2=0x7fcdbc1068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:24.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.445+0000 7fcdc15d9700 1 --2- 172.21.15.6:0/3097938929 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdbc102860 0x7fcdbc1068b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009990 tx=0x7fcdb0009ca0 comp rx=0 tx=0).stop 2024-03-24T12:10:24.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.446+0000 7fcdc15d9700 1 -- 172.21.15.6:0/3097938929 shutdown_connections 2024-03-24T12:10:24.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.446+0000 7fcdc15d9700 1 --2- 172.21.15.6:0/3097938929 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcdbc106f90 0x7fcdbc10eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:24.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.446+0000 7fcdc15d9700 1 --2- 172.21.15.6:0/3097938929 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdbc101eb0 0x7fcdbc102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:24.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.446+0000 7fcdc15d9700 1 --2- 172.21.15.6:0/3097938929 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdbc102860 0x7fcdbc1068b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:24.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.446+0000 7fcdc15d9700 1 -- 172.21.15.6:0/3097938929 >> 172.21.15.6:0/3097938929 conn(0x7fcdbc0fd720 msgr2=0x7fcdbc0ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:24.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.446+0000 7fcdc15d9700 1 -- 172.21.15.6:0/3097938929 shutdown_connections 2024-03-24T12:10:24.959 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.446+0000 7fcdc15d9700 1 -- 172.21.15.6:0/3097938929 wait complete. 2024-03-24T12:10:24.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.446+0000 7fcdc15d9700 1 Processor -- start 2024-03-24T12:10:24.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.447+0000 7fcdc15d9700 1 -- start start 2024-03-24T12:10:24.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.447+0000 7fcdc15d9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdbc101eb0 0x7fcdbc073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:24.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.447+0000 7fcdc15d9700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdbc102860 0x7fcdbc0744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:24.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.447+0000 7fcdc15d9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcdbc106f90 0x7fcdbc0749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:24.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.447+0000 7fcdc15d9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc1a5b90 con 0x7fcdbc101eb0 2024-03-24T12:10:24.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.447+0000 7fcdc15d9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc1a5d00 con 0x7fcdbc106f90 2024-03-24T12:10:24.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.447+0000 7fcdc15d9700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc1a5ec0 con 0x7fcdbc102860 2024-03-24T12:10:24.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.447+0000 7fcdbaffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdbc101eb0 0x7fcdbc073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:24.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.447+0000 7fcdba7fc700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdbc102860 0x7fcdbc0744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:24.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.447+0000 7fcdbaffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdbc101eb0 0x7fcdbc073f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58838/0 (socket says 172.21.15.6:58838) 2024-03-24T12:10:24.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.447+0000 7fcdbaffd700 1 -- 172.21.15.6:0/2215189404 learned_addr learned my addr 172.21.15.6:0/2215189404 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:10:24.960 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.447+0000 7fcdbb7fe700 1 --2- 172.21.15.6:0/2215189404 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcdbc106f90 0x7fcdbc0749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:24.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.447+0000 7fcdbaffd700 1 -- 172.21.15.6:0/2215189404 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdbc102860 msgr2=0x7fcdbc0744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:24.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.447+0000 7fcdbaffd700 1 --2- 172.21.15.6:0/2215189404 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdbc102860 0x7fcdbc0744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:24.961 INFO:teuthology.orchestra.run.smithi006.stdout:128849018912 2024-03-24T12:10:24.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.447+0000 7fcdbaffd700 1 -- 172.21.15.6:0/2215189404 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcdbc106f90 msgr2=0x7fcdbc0749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:24.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.447+0000 7fcdbaffd700 1 --2- 172.21.15.6:0/2215189404 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcdbc106f90 0x7fcdbc0749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:24.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.447+0000 7fcdbaffd700 1 -- 172.21.15.6:0/2215189404 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac009520 con 0x7fcdbc101eb0 2024-03-24T12:10:24.961 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.447+0000 7fcdba7fc700 1 --2- 172.21.15.6:0/2215189404 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdbc102860 0x7fcdbc0744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:10:24.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.448+0000 7fcdbaffd700 1 --2- 172.21.15.6:0/2215189404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdbc101eb0 0x7fcdbc073f70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00eb30 tx=0x7fcdac00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:24.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.448+0000 7fcda3fff700 1 -- 172.21.15.6:0/2215189404 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac009880 con 0x7fcdbc101eb0 2024-03-24T12:10:24.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.448+0000 7fcda3fff700 1 -- 172.21.15.6:0/2215189404 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fcdac0044a0 con 0x7fcdbc101eb0 2024-03-24T12:10:24.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.448+0000 7fcdc15d9700 1 -- 172.21.15.6:0/2215189404 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb00095f0 con 0x7fcdbc101eb0 2024-03-24T12:10:24.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.448+0000 7fcda3fff700 1 -- 172.21.15.6:0/2215189404 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac003d10 con 0x7fcdbc101eb0 2024-03-24T12:10:24.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.448+0000 7fcdc15d9700 1 -- 172.21.15.6:0/2215189404 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdbc1a64e0 con 0x7fcdbc101eb0 2024-03-24T12:10:24.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.449+0000 7fcdc15d9700 1 -- 172.21.15.6:0/2215189404 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdbc068930 con 0x7fcdbc101eb0 2024-03-24T12:10:24.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.449+0000 7fcda3fff700 1 -- 172.21.15.6:0/2215189404 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7fcdac004020 con 0x7fcdbc101eb0 2024-03-24T12:10:24.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.450+0000 7fcda3fff700 1 --2- 172.21.15.6:0/2215189404 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fcda406d660 0x7fcda406fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:24.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.450+0000 7fcda3fff700 1 -- 172.21.15.6:0/2215189404 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7fcdac014070 con 0x7fcdbc101eb0 2024-03-24T12:10:24.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.450+0000 7fcdba7fc700 1 --2- 172.21.15.6:0/2215189404 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fcda406d660 0x7fcda406fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:24.962 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.451+0000 7fcdba7fc700 1 --2- 172.21.15.6:0/2215189404 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fcda406d660 0x7fcda406fb20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00058e0 tx=0x7fcdb0005960 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:24.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.455+0000 7fcda3fff700 1 -- 172.21.15.6:0/2215189404 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcdac056d30 con 0x7fcdbc101eb0 2024-03-24T12:10:24.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.665+0000 7fcdc15d9700 1 -- 172.21.15.6:0/2215189404 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fcdbc111210 con 0x7fcdbc101eb0 2024-03-24T12:10:24.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.665+0000 7fcda3fff700 1 -- 172.21.15.6:0/2215189404 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcdac07a090 con 0x7fcdbc101eb0 2024-03-24T12:10:24.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.667+0000 7fcdc15d9700 1 -- 172.21.15.6:0/2215189404 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fcda406d660 msgr2=0x7fcda406fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:24.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.667+0000 7fcdc15d9700 1 --2- 172.21.15.6:0/2215189404 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fcda406d660 0x7fcda406fb20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00058e0 tx=0x7fcdb0005960 comp rx=0 tx=0).stop 2024-03-24T12:10:24.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.668+0000 7fcdc15d9700 1 -- 172.21.15.6:0/2215189404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdbc101eb0 msgr2=0x7fcdbc073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:24.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.668+0000 7fcdc15d9700 1 --2- 172.21.15.6:0/2215189404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdbc101eb0 0x7fcdbc073f70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00eb30 tx=0x7fcdac00ee40 comp rx=0 tx=0).stop 2024-03-24T12:10:24.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.668+0000 7fcdc15d9700 1 -- 172.21.15.6:0/2215189404 shutdown_connections 2024-03-24T12:10:24.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.668+0000 7fcdc15d9700 1 --2- 172.21.15.6:0/2215189404 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcdbc106f90 0x7fcdbc0749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:24.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.668+0000 7fcdc15d9700 1 --2- 172.21.15.6:0/2215189404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcdbc101eb0 0x7fcdbc073f70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:24.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.668+0000 7fcdc15d9700 1 --2- 172.21.15.6:0/2215189404 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fcda406d660 0x7fcda406fb20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:24.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.668+0000 7fcdc15d9700 1 --2- 172.21.15.6:0/2215189404 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcdbc102860 0x7fcdbc0744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:24.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.668+0000 7fcdc15d9700 1 -- 172.21.15.6:0/2215189404 >> 172.21.15.6:0/2215189404 conn(0x7fcdbc0fd720 msgr2=0x7fcdbc0fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:24.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.668+0000 7fcdc15d9700 1 -- 172.21.15.6:0/2215189404 shutdown_connections 2024-03-24T12:10:24.963 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.668+0000 7fcdc15d9700 1 -- 172.21.15.6:0/2215189404 wait complete. 2024-03-24T12:10:25.461 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:10:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:10:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:10:25.662 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018908 got 128849018912 for osd.4 2024-03-24T12:10:25.663 DEBUG:teuthology.parallel:result is None 2024-03-24T12:10:25.764 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2215189404' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-24T12:10:25.764 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2228782607' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-24T12:10:25.765 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2215189404' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-24T12:10:25.765 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2228782607' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-24T12:10:25.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2215189404' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-24T12:10:25.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2228782607' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-24T12:10:26.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.974+0000 7ff5205c0700 1 -- 172.21.15.6:0/2841643758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5181011c0 msgr2=0x7ff518106900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:26.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.974+0000 7ff5205c0700 1 --2- 172.21.15.6:0/2841643758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5181011c0 0x7ff518106900 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009990 tx=0x7ff50c009ca0 comp rx=0 tx=0).stop 2024-03-24T12:10:26.007 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.974+0000 7ff5205c0700 1 -- 172.21.15.6:0/2841643758 shutdown_connections 2024-03-24T12:10:26.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.974+0000 7ff5205c0700 1 --2- 172.21.15.6:0/2841643758 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff518107030 0x7ff51810ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:26.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.974+0000 7ff5205c0700 1 --2- 172.21.15.6:0/2841643758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff518100810 0x7ff518100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:26.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.974+0000 7ff5205c0700 1 --2- 172.21.15.6:0/2841643758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5181011c0 0x7ff518106900 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:26.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.974+0000 7ff5205c0700 1 -- 172.21.15.6:0/2841643758 >> 172.21.15.6:0/2841643758 conn(0x7ff518075eb0 msgr2=0x7ff5180762c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:26.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.974+0000 7ff5205c0700 1 -- 172.21.15.6:0/2841643758 shutdown_connections 2024-03-24T12:10:26.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.974+0000 7ff5205c0700 1 -- 172.21.15.6:0/2841643758 wait complete. 2024-03-24T12:10:26.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.974+0000 7ff5205c0700 1 Processor -- start 2024-03-24T12:10:26.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.975+0000 7ff5205c0700 1 -- start start 2024-03-24T12:10:26.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.975+0000 7ff5205c0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff518100810 0x7ff518197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:26.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.975+0000 7ff5205c0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5181011c0 0x7ff518197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:26.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.975+0000 7ff5205c0700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff518107030 0x7ff51819c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:26.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.975+0000 7ff5205c0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51819c630 con 0x7ff518100810 2024-03-24T12:10:26.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.975+0000 7ff5205c0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51819c7a0 con 0x7ff5181011c0 2024-03-24T12:10:26.008 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.975+0000 7ff5205c0700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51819c960 con 0x7ff518107030 2024-03-24T12:10:26.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.975+0000 7ff51e35c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff518100810 0x7ff518197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:26.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.975+0000 7ff51eb5d700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff518107030 0x7ff51819c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:26.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.975+0000 7ff51e35c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff518100810 0x7ff518197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58854/0 (socket says 172.21.15.6:58854) 2024-03-24T12:10:26.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.975+0000 7ff51e35c700 1 -- 172.21.15.6:0/2228782607 learned_addr learned my addr 172.21.15.6:0/2228782607 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:10:26.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.975+0000 7ff51db5b700 1 --2- 172.21.15.6:0/2228782607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5181011c0 0x7ff518197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:26.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.975+0000 7ff51eb5d700 1 -- 172.21.15.6:0/2228782607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5181011c0 msgr2=0x7ff518197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:26.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.975+0000 7ff51eb5d700 1 --2- 172.21.15.6:0/2228782607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5181011c0 0x7ff518197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:26.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.975+0000 7ff51eb5d700 1 -- 172.21.15.6:0/2228782607 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff518100810 msgr2=0x7ff518197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:26.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.975+0000 7ff51eb5d700 1 --2- 172.21.15.6:0/2228782607 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff518100810 0x7ff518197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:26.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.975+0000 7ff51eb5d700 1 -- 172.21.15.6:0/2228782607 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff508009520 con 0x7ff518107030 2024-03-24T12:10:26.009 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.975+0000 7ff51e35c700 1 --2- 172.21.15.6:0/2228782607 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff518100810 0x7ff518197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:10:26.010 INFO:teuthology.orchestra.run.smithi006.stdout:163208757275 2024-03-24T12:10:26.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.975+0000 7ff51eb5d700 1 --2- 172.21.15.6:0/2228782607 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff518107030 0x7ff51819c020 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff50800e900 tx=0x7ff50800ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:26.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.975+0000 7ff5177fe700 1 -- 172.21.15.6:0/2228782607 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508004220 con 0x7ff518107030 2024-03-24T12:10:26.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.976+0000 7ff5177fe700 1 -- 172.21.15.6:0/2228782607 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ff508004380 con 0x7ff518107030 2024-03-24T12:10:26.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.976+0000 7ff5205c0700 1 -- 172.21.15.6:0/2228782607 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50c0095f0 con 0x7ff518107030 2024-03-24T12:10:26.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.976+0000 7ff5177fe700 1 -- 172.21.15.6:0/2228782607 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508003b60 con 0x7ff518107030 2024-03-24T12:10:26.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.976+0000 7ff5205c0700 1 -- 172.21.15.6:0/2228782607 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5181a42d0 con 0x7ff518107030 2024-03-24T12:10:26.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.976+0000 7ff5205c0700 1 -- 172.21.15.6:0/2228782607 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff51810a270 con 0x7ff518107030 2024-03-24T12:10:26.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.977+0000 7ff5177fe700 1 -- 172.21.15.6:0/2228782607 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7ff508003d40 con 0x7ff518107030 2024-03-24T12:10:26.010 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.977+0000 7ff5177fe700 1 --2- 172.21.15.6:0/2228782607 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7ff50006d660 0x7ff50006fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.977+0000 7ff5177fe700 1 -- 172.21.15.6:0/2228782607 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7ff508014070 con 0x7ff518107030 2024-03-24T12:10:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.977+0000 7ff51e35c700 1 --2- 172.21.15.6:0/2228782607 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7ff50006d660 0x7ff50006fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.978+0000 7ff51e35c700 1 --2- 172.21.15.6:0/2228782607 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7ff50006d660 0x7ff50006fb20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff510009fd0 tx=0x7ff510009040 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:24.982+0000 7ff5177fe700 1 -- 172.21.15.6:0/2228782607 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff508056b00 con 0x7ff518107030 2024-03-24T12:10:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:25.186+0000 7ff5205c0700 1 -- 172.21.15.6:0/2228782607 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ff518198ab0 con 0x7ff518107030 2024-03-24T12:10:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:25.187+0000 7ff5177fe700 1 -- 172.21.15.6:0/2228782607 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff50805a1c0 con 0x7ff518107030 2024-03-24T12:10:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:25.189+0000 7ff5205c0700 1 -- 172.21.15.6:0/2228782607 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7ff50006d660 msgr2=0x7ff50006fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:25.189+0000 7ff5205c0700 1 --2- 172.21.15.6:0/2228782607 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7ff50006d660 0x7ff50006fb20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff510009fd0 tx=0x7ff510009040 comp rx=0 tx=0).stop 2024-03-24T12:10:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:25.189+0000 7ff5205c0700 1 -- 172.21.15.6:0/2228782607 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff518107030 msgr2=0x7ff51819c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:25.189+0000 7ff5205c0700 1 --2- 172.21.15.6:0/2228782607 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff518107030 0x7ff51819c020 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff50800e900 tx=0x7ff50800ec10 comp rx=0 tx=0).stop 2024-03-24T12:10:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:25.189+0000 7ff5205c0700 1 -- 172.21.15.6:0/2228782607 shutdown_connections 2024-03-24T12:10:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:25.189+0000 7ff5205c0700 1 --2- 172.21.15.6:0/2228782607 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7ff50006d660 0x7ff50006fb20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:25.189+0000 7ff5205c0700 1 --2- 172.21.15.6:0/2228782607 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff518107030 0x7ff51819c020 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:25.189+0000 7ff5205c0700 1 --2- 172.21.15.6:0/2228782607 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff518100810 0x7ff518197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:25.189+0000 7ff5205c0700 1 --2- 172.21.15.6:0/2228782607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5181011c0 0x7ff518197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:26.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:25.189+0000 7ff5205c0700 1 -- 172.21.15.6:0/2228782607 >> 172.21.15.6:0/2228782607 conn(0x7ff518075eb0 msgr2=0x7ff5181081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:26.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:25.189+0000 7ff5205c0700 1 -- 172.21.15.6:0/2228782607 shutdown_connections 2024-03-24T12:10:26.012 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:25.189+0000 7ff5205c0700 1 -- 172.21.15.6:0/2228782607 wait complete. 2024-03-24T12:10:26.536 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:26 smithi006 ceph-mon[30108]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:26.537 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:26 smithi006 ceph-mon[41734]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:26.566 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757272 got 163208757275 for osd.5 2024-03-24T12:10:26.566 DEBUG:teuthology.parallel:result is None 2024-03-24T12:10:26.829 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:26 smithi090 ceph-mon[33646]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:27.678 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495635 got 197568495638 for osd.6 2024-03-24T12:10:27.678 DEBUG:teuthology.parallel:result is None 2024-03-24T12:10:28.152 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:10:28.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.168+0000 7f0356418700 1 -- 172.21.15.6:0/2825527451 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0350100ba0 msgr2=0x7f0350101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:28.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.168+0000 7f0356418700 1 --2- 172.21.15.6:0/2825527451 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0350100ba0 0x7f0350101020 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0344009990 tx=0x7f0344009ca0 comp rx=0 tx=0).stop 2024-03-24T12:10:28.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.168+0000 7f0356418700 1 -- 172.21.15.6:0/2825527451 shutdown_connections 2024-03-24T12:10:28.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.168+0000 7f0356418700 1 --2- 172.21.15.6:0/2825527451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0350101560 0x7f035010d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:28.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.168+0000 7f0356418700 1 --2- 172.21.15.6:0/2825527451 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0350106bd0 0x7f0350106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:28.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.168+0000 7f0356418700 1 --2- 172.21.15.6:0/2825527451 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0350100ba0 0x7f0350101020 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:28.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.168+0000 7f0356418700 1 -- 172.21.15.6:0/2825527451 >> 172.21.15.6:0/2825527451 conn(0x7f0350076bc0 msgr2=0x7f0350076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:28.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.168+0000 7f0356418700 1 -- 172.21.15.6:0/2825527451 shutdown_connections 2024-03-24T12:10:28.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.168+0000 7f0356418700 1 -- 172.21.15.6:0/2825527451 wait complete. 2024-03-24T12:10:28.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.169+0000 7f0356418700 1 Processor -- start 2024-03-24T12:10:28.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.169+0000 7f0356418700 1 -- start start 2024-03-24T12:10:28.170 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.169+0000 7f0356418700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0350100ba0 0x7f03501976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:28.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.169+0000 7f0356418700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0350101560 0x7f0350197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:28.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.169+0000 7f0356418700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0350106bd0 0x7f035019bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:28.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.169+0000 7f0356418700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035019c580 con 0x7f0350101560 2024-03-24T12:10:28.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.169+0000 7f0356418700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035019c6f0 con 0x7f0350106bd0 2024-03-24T12:10:28.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.169+0000 7f0356418700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035019c8b0 con 0x7f0350100ba0 2024-03-24T12:10:28.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.170+0000 7f034ffff700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0350100ba0 0x7f03501976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:28.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.170+0000 7f034ffff700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0350100ba0 0x7f03501976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:35816/0 (socket says 172.21.15.6:35816) 2024-03-24T12:10:28.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.170+0000 7f034ffff700 1 -- 172.21.15.6:0/966655470 learned_addr learned my addr 172.21.15.6:0/966655470 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:10:28.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.170+0000 7f034f7fe700 1 --2- 172.21.15.6:0/966655470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0350101560 0x7f0350197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:28.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.170+0000 7f03549b5700 1 --2- 172.21.15.6:0/966655470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0350106bd0 0x7f035019bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:28.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.170+0000 7f034ffff700 1 -- 172.21.15.6:0/966655470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0350106bd0 msgr2=0x7f035019bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:28.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.170+0000 7f034ffff700 1 --2- 172.21.15.6:0/966655470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0350106bd0 0x7f035019bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:28.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.170+0000 7f034ffff700 1 -- 172.21.15.6:0/966655470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0350101560 msgr2=0x7f0350197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:28.171 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.170+0000 7f034ffff700 1 --2- 172.21.15.6:0/966655470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0350101560 0x7f0350197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:28.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.170+0000 7f034ffff700 1 -- 172.21.15.6:0/966655470 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03440095f0 con 0x7f0350100ba0 2024-03-24T12:10:28.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.170+0000 7f03549b5700 1 --2- 172.21.15.6:0/966655470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0350106bd0 0x7f035019bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T12:10:28.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.170+0000 7f034ffff700 1 --2- 172.21.15.6:0/966655470 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0350100ba0 0x7f03501976a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f034000e9a0 tx=0x7f034000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:28.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.170+0000 7f034d7fa700 1 -- 172.21.15.6:0/966655470 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0340009790 con 0x7f0350100ba0 2024-03-24T12:10:28.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.170+0000 7f034d7fa700 1 -- 172.21.15.6:0/966655470 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f034000cbd0 con 0x7f0350100ba0 2024-03-24T12:10:28.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.170+0000 7f034d7fa700 1 -- 172.21.15.6:0/966655470 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0340018420 con 0x7f0350100ba0 2024-03-24T12:10:28.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.170+0000 7f0356418700 1 -- 172.21.15.6:0/966655470 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f035019cb90 con 0x7f0350100ba0 2024-03-24T12:10:28.172 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.170+0000 7f0356418700 1 -- 172.21.15.6:0/966655470 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0350102890 con 0x7f0350100ba0 2024-03-24T12:10:28.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.172+0000 7f0356418700 1 -- 172.21.15.6:0/966655470 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03501021d0 con 0x7f0350100ba0 2024-03-24T12:10:28.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.172+0000 7f034d7fa700 1 -- 172.21.15.6:0/966655470 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7f0340010430 con 0x7f0350100ba0 2024-03-24T12:10:28.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.173+0000 7f034d7fa700 1 --2- 172.21.15.6:0/966655470 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f033806d660 0x7f033806fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:28.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.173+0000 7f034d7fa700 1 -- 172.21.15.6:0/966655470 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f0340014070 con 0x7f0350100ba0 2024-03-24T12:10:28.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.173+0000 7f034f7fe700 1 --2- 172.21.15.6:0/966655470 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f033806d660 0x7f033806fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:28.173 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.173+0000 7f034f7fe700 1 --2- 172.21.15.6:0/966655470 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f033806d660 0x7f033806fb20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f034400b6f0 tx=0x7f034400b280 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:28.177 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.178+0000 7f034d7fa700 1 -- 172.21.15.6:0/966655470 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0340052d20 con 0x7f0350100ba0 2024-03-24T12:10:28.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.385+0000 7f0356418700 1 -- 172.21.15.6:0/966655470 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0350198a00 con 0x7f0350100ba0 2024-03-24T12:10:28.384 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.385+0000 7f034d7fa700 1 -- 172.21.15.6:0/966655470 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0340056b80 con 0x7f0350100ba0 2024-03-24T12:10:28.385 INFO:teuthology.orchestra.run.smithi006.stdout:77309411372 2024-03-24T12:10:28.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.387+0000 7f0356418700 1 -- 172.21.15.6:0/966655470 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f033806d660 msgr2=0x7f033806fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:28.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.387+0000 7f0356418700 1 --2- 172.21.15.6:0/966655470 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f033806d660 0x7f033806fb20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f034400b6f0 tx=0x7f034400b280 comp rx=0 tx=0).stop 2024-03-24T12:10:28.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.387+0000 7f0356418700 1 -- 172.21.15.6:0/966655470 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0350100ba0 msgr2=0x7f03501976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:28.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.387+0000 7f0356418700 1 --2- 172.21.15.6:0/966655470 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0350100ba0 0x7f03501976a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f034000e9a0 tx=0x7f034000ecb0 comp rx=0 tx=0).stop 2024-03-24T12:10:28.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.387+0000 7f0356418700 1 -- 172.21.15.6:0/966655470 shutdown_connections 2024-03-24T12:10:28.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.387+0000 7f0356418700 1 --2- 172.21.15.6:0/966655470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0350106bd0 0x7f035019bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:28.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.387+0000 7f0356418700 1 --2- 172.21.15.6:0/966655470 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f033806d660 0x7f033806fb20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:28.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.387+0000 7f0356418700 1 --2- 172.21.15.6:0/966655470 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0350100ba0 0x7f03501976a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:28.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.387+0000 7f0356418700 1 --2- 172.21.15.6:0/966655470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0350101560 0x7f0350197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:28.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.387+0000 7f0356418700 1 -- 172.21.15.6:0/966655470 >> 172.21.15.6:0/966655470 conn(0x7f0350076bc0 msgr2=0x7f03500ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:28.387 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.388+0000 7f0356418700 1 -- 172.21.15.6:0/966655470 shutdown_connections 2024-03-24T12:10:28.388 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:28.388+0000 7f0356418700 1 -- 172.21.15.6:0/966655470 wait complete. 2024-03-24T12:10:28.684 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:28 smithi006 ceph-mon[30108]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:10:28.684 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/966655470' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-24T12:10:28.684 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:28 smithi006 ceph-mon[41734]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:10:28.684 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/966655470' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-24T12:10:28.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:28 smithi090 ceph-mon[33646]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:10:28.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/966655470' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-24T12:10:29.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.205+0000 7f4e101b7700 1 -- 172.21.15.6:0/1562989133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e08100f40 msgr2=0x7f4e08104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:29.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.205+0000 7f4e101b7700 1 --2- 172.21.15.6:0/1562989133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e08100f40 0x7f4e08104f90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009990 tx=0x7f4df8009ca0 comp rx=0 tx=0).stop 2024-03-24T12:10:29.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.205+0000 7f4e101b7700 1 -- 172.21.15.6:0/1562989133 shutdown_connections 2024-03-24T12:10:29.219 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.205+0000 7f4e101b7700 1 --2- 172.21.15.6:0/1562989133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e08105670 0x7f4e0810d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:29.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.205+0000 7f4e101b7700 1 --2- 172.21.15.6:0/1562989133 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e08100590 0x7f4e08100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:29.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.205+0000 7f4e101b7700 1 --2- 172.21.15.6:0/1562989133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e08100f40 0x7f4e08104f90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:29.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.205+0000 7f4e101b7700 1 -- 172.21.15.6:0/1562989133 >> 172.21.15.6:0/1562989133 conn(0x7f4e08077170 msgr2=0x7f4e080fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:29.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.205+0000 7f4e101b7700 1 -- 172.21.15.6:0/1562989133 shutdown_connections 2024-03-24T12:10:29.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.205+0000 7f4e101b7700 1 -- 172.21.15.6:0/1562989133 wait complete. 2024-03-24T12:10:29.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.205+0000 7f4e101b7700 1 Processor -- start 2024-03-24T12:10:29.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.205+0000 7f4e101b7700 1 -- start start 2024-03-24T12:10:29.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4e101b7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e08100590 0x7f4e0819d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:29.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4e101b7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e08100f40 0x7f4e0819d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:29.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4e101b7700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e08105670 0x7f4e081975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:29.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4e101b7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e08197c30 con 0x7f4e08100590 2024-03-24T12:10:29.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4e101b7700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e08197d70 con 0x7f4e08100f40 2024-03-24T12:10:29.220 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4e101b7700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e08197f30 con 0x7f4e08105670 2024-03-24T12:10:29.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4e0df53700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e08100590 0x7f4e0819d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:29.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4e0e754700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e08105670 0x7f4e081975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:29.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4e0df53700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e08100590 0x7f4e0819d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37604/0 (socket says 172.21.15.6:37604) 2024-03-24T12:10:29.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4e0e754700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e08105670 0x7f4e081975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:35834/0 (socket says 172.21.15.6:35834) 2024-03-24T12:10:29.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4e0df53700 1 -- 172.21.15.6:0/3971397626 learned_addr learned my addr 172.21.15.6:0/3971397626 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:10:29.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4e0d752700 1 --2- 172.21.15.6:0/3971397626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e08100f40 0x7f4e0819d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:29.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4e0e754700 1 -- 172.21.15.6:0/3971397626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e08100f40 msgr2=0x7f4e0819d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:29.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4e0e754700 1 --2- 172.21.15.6:0/3971397626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e08100f40 0x7f4e0819d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:29.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4e0e754700 1 -- 172.21.15.6:0/3971397626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e08100590 msgr2=0x7f4e0819d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:29.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4e0e754700 1 --2- 172.21.15.6:0/3971397626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e08100590 0x7f4e0819d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:29.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4e0e754700 1 -- 172.21.15.6:0/3971397626 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0400a630 con 0x7f4e08105670 2024-03-24T12:10:29.221 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4e0df53700 1 --2- 172.21.15.6:0/3971397626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e08100590 0x7f4e0819d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:10:29.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4e0e754700 1 --2- 172.21.15.6:0/3971397626 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e08105670 0x7f4e081975d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4e00007f10 tx=0x7f4e0000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:29.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4dfeffd700 1 -- 172.21.15.6:0/3971397626 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0000ddd0 con 0x7f4e08105670 2024-03-24T12:10:29.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4e101b7700 1 -- 172.21.15.6:0/3971397626 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df80095f0 con 0x7f4e08105670 2024-03-24T12:10:29.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4dfeffd700 1 -- 172.21.15.6:0/3971397626 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f4e0000f040 con 0x7f4e08105670 2024-03-24T12:10:29.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4dfeffd700 1 -- 172.21.15.6:0/3971397626 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e00005930 con 0x7f4e08105670 2024-03-24T12:10:29.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.206+0000 7f4e101b7700 1 -- 172.21.15.6:0/3971397626 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e081a42e0 con 0x7f4e08105670 2024-03-24T12:10:29.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.207+0000 7f4e101b7700 1 -- 172.21.15.6:0/3971397626 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e08101a70 con 0x7f4e08105670 2024-03-24T12:10:29.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.208+0000 7f4dfeffd700 1 -- 172.21.15.6:0/3971397626 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7f4e00004ad0 con 0x7f4e08105670 2024-03-24T12:10:29.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.208+0000 7f4dfeffd700 1 --2- 172.21.15.6:0/3971397626 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f4df406d660 0x7f4df406fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:29.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.209+0000 7f4e0df53700 1 --2- 172.21.15.6:0/3971397626 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f4df406d660 0x7f4df406fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:29.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.209+0000 7f4dfeffd700 1 -- 172.21.15.6:0/3971397626 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f4e0008cb50 con 0x7f4e08105670 2024-03-24T12:10:29.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.209+0000 7f4e0df53700 1 --2- 172.21.15.6:0/3971397626 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f4df406d660 0x7f4df406fb20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400e740 tx=0x7f4e0400a370 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:29.222 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.213+0000 7f4dfeffd700 1 -- 172.21.15.6:0/3971397626 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4e000561a0 con 0x7f4e08105670 2024-03-24T12:10:29.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.426+0000 7f4e101b7700 1 -- 172.21.15.6:0/3971397626 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4e08198b00 con 0x7f4e08105670 2024-03-24T12:10:29.426 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.426+0000 7f4dfeffd700 1 -- 172.21.15.6:0/3971397626 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4e00059860 con 0x7f4e08105670 2024-03-24T12:10:29.426 INFO:teuthology.orchestra.run.smithi006.stdout:55834574897 2024-03-24T12:10:29.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.428+0000 7f4e101b7700 1 -- 172.21.15.6:0/3971397626 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f4df406d660 msgr2=0x7f4df406fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:29.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.428+0000 7f4e101b7700 1 --2- 172.21.15.6:0/3971397626 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f4df406d660 0x7f4df406fb20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400e740 tx=0x7f4e0400a370 comp rx=0 tx=0).stop 2024-03-24T12:10:29.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.428+0000 7f4e101b7700 1 -- 172.21.15.6:0/3971397626 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e08105670 msgr2=0x7f4e081975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:29.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.428+0000 7f4e101b7700 1 --2- 172.21.15.6:0/3971397626 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e08105670 0x7f4e081975d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4e00007f10 tx=0x7f4e0000d3b0 comp rx=0 tx=0).stop 2024-03-24T12:10:29.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.428+0000 7f4e101b7700 1 -- 172.21.15.6:0/3971397626 shutdown_connections 2024-03-24T12:10:29.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.428+0000 7f4e101b7700 1 --2- 172.21.15.6:0/3971397626 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f4df406d660 0x7f4df406fb20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:29.428 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.428+0000 7f4e101b7700 1 --2- 172.21.15.6:0/3971397626 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e08105670 0x7f4e081975d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:29.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.428+0000 7f4e101b7700 1 --2- 172.21.15.6:0/3971397626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e08100590 0x7f4e0819d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:29.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.428+0000 7f4e101b7700 1 --2- 172.21.15.6:0/3971397626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e08100f40 0x7f4e0819d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:29.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.428+0000 7f4e101b7700 1 -- 172.21.15.6:0/3971397626 >> 172.21.15.6:0/3971397626 conn(0x7f4e08077170 msgr2=0x7f4e081041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:29.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.428+0000 7f4e101b7700 1 -- 172.21.15.6:0/3971397626 shutdown_connections 2024-03-24T12:10:29.429 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.429+0000 7f4e101b7700 1 -- 172.21.15.6:0/3971397626 wait complete. 2024-03-24T12:10:29.456 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.455+0000 7fafb037c700 1 -- 172.21.15.6:0/712080603 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fafa8107cc0 msgr2=0x7fafa81080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:29.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.455+0000 7fafb037c700 1 --2- 172.21.15.6:0/712080603 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fafa8107cc0 0x7fafa81080a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fafa40099c0 tx=0x7fafa4009cd0 comp rx=0 tx=0).stop 2024-03-24T12:10:29.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.455+0000 7fafb037c700 1 -- 172.21.15.6:0/712080603 shutdown_connections 2024-03-24T12:10:29.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.455+0000 7fafb037c700 1 --2- 172.21.15.6:0/712080603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafa806a2c0 0x7fafa810eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:29.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.455+0000 7fafb037c700 1 --2- 172.21.15.6:0/712080603 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fafa8107cc0 0x7fafa81080a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:29.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.455+0000 7fafb037c700 1 --2- 172.21.15.6:0/712080603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafa8069920 0x7fafa8069d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:29.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.455+0000 7fafb037c700 1 -- 172.21.15.6:0/712080603 >> 172.21.15.6:0/712080603 conn(0x7fafa8077aa0 msgr2=0x7fafa8077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:29.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.455+0000 7fafb037c700 1 -- 172.21.15.6:0/712080603 shutdown_connections 2024-03-24T12:10:29.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.455+0000 7fafb037c700 1 -- 172.21.15.6:0/712080603 wait complete. 2024-03-24T12:10:29.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.456+0000 7fafb037c700 1 Processor -- start 2024-03-24T12:10:29.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.456+0000 7fafb037c700 1 -- start start 2024-03-24T12:10:29.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.456+0000 7fafb037c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafa8069920 0x7fafa8100c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:29.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.456+0000 7fafb037c700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fafa806a2c0 0x7fafa8101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:29.457 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.456+0000 7fafb037c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafa8107cc0 0x7fafa81016c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:29.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.456+0000 7fafb037c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa81a5b50 con 0x7fafa8069920 2024-03-24T12:10:29.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.456+0000 7fafae118700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafa8069920 0x7fafa8100c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:29.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.456+0000 7fafae118700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafa8069920 0x7fafa8100c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:37612/0 (socket says 172.21.15.6:37612) 2024-03-24T12:10:29.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.456+0000 7fafb037c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa81a5cc0 con 0x7fafa8107cc0 2024-03-24T12:10:29.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.456+0000 7fafad917700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fafa806a2c0 0x7fafa8101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:29.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.456+0000 7fafae118700 1 -- 172.21.15.6:0/1365325782 learned_addr learned my addr 172.21.15.6:0/1365325782 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:10:29.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.456+0000 7fafb037c700 1 -- 172.21.15.6:0/1365325782 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa81a5e80 con 0x7fafa806a2c0 2024-03-24T12:10:29.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.456+0000 7fafae919700 1 --2- 172.21.15.6:0/1365325782 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafa8107cc0 0x7fafa81016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:29.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.457+0000 7fafae118700 1 -- 172.21.15.6:0/1365325782 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fafa806a2c0 msgr2=0x7fafa8101180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:29.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.457+0000 7fafae118700 1 --2- 172.21.15.6:0/1365325782 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fafa806a2c0 0x7fafa8101180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:29.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.457+0000 7fafae118700 1 -- 172.21.15.6:0/1365325782 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafa8107cc0 msgr2=0x7fafa81016c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:29.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.457+0000 7fafae118700 1 --2- 172.21.15.6:0/1365325782 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafa8107cc0 0x7fafa81016c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:29.458 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.457+0000 7fafae118700 1 -- 172.21.15.6:0/1365325782 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa0009c40 con 0x7fafa8069920 2024-03-24T12:10:29.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.457+0000 7fafad917700 1 --2- 172.21.15.6:0/1365325782 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fafa806a2c0 0x7fafa8101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:10:29.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.457+0000 7fafae118700 1 --2- 172.21.15.6:0/1365325782 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafa8069920 0x7fafa8100c40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fafa400b580 tx=0x7fafa400b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:29.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.457+0000 7faf9f7fe700 1 -- 172.21.15.6:0/1365325782 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa401d070 con 0x7fafa8069920 2024-03-24T12:10:29.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.457+0000 7faf9f7fe700 1 -- 172.21.15.6:0/1365325782 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fafa4004cb0 con 0x7fafa8069920 2024-03-24T12:10:29.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.457+0000 7faf9f7fe700 1 -- 172.21.15.6:0/1365325782 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa4003680 con 0x7fafa8069920 2024-03-24T12:10:29.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.457+0000 7fafb037c700 1 -- 172.21.15.6:0/1365325782 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa40095f0 con 0x7fafa8069920 2024-03-24T12:10:29.459 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.457+0000 7fafb037c700 1 -- 172.21.15.6:0/1365325782 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa81a62d0 con 0x7fafa8069920 2024-03-24T12:10:29.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.459+0000 7fafb037c700 1 -- 172.21.15.6:0/1365325782 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafa810a1e0 con 0x7fafa8069920 2024-03-24T12:10:29.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.459+0000 7faf9f7fe700 1 -- 172.21.15.6:0/1365325782 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7fafa4004830 con 0x7fafa8069920 2024-03-24T12:10:29.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.459+0000 7faf9f7fe700 1 --2- 172.21.15.6:0/1365325782 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7faf9406d660 0x7faf9406fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:29.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.459+0000 7faf9f7fe700 1 -- 172.21.15.6:0/1365325782 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7fafa408ddd0 con 0x7fafa8069920 2024-03-24T12:10:29.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.460+0000 7fafad917700 1 --2- 172.21.15.6:0/1365325782 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7faf9406d660 0x7faf9406fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:29.460 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.460+0000 7fafad917700 1 --2- 172.21.15.6:0/1365325782 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7faf9406d660 0x7faf9406fb20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fafa80780f0 tx=0x7faf98009260 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:29.463 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.464+0000 7faf9f7fe700 1 -- 172.21.15.6:0/1365325782 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fafa40574a0 con 0x7fafa8069920 2024-03-24T12:10:29.667 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.668+0000 7fafb037c700 1 -- 172.21.15.6:0/1365325782 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fafa8102170 con 0x7fafa8069920 2024-03-24T12:10:29.668 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:29 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3971397626' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-24T12:10:29.668 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:29 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3971397626' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-24T12:10:29.668 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.668+0000 7faf9f7fe700 1 -- 172.21.15.6:0/1365325782 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fafa405ab60 con 0x7fafa8069920 2024-03-24T12:10:29.668 INFO:teuthology.orchestra.run.smithi006.stdout:34359738421 2024-03-24T12:10:29.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.670+0000 7fafb037c700 1 -- 172.21.15.6:0/1365325782 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7faf9406d660 msgr2=0x7faf9406fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:29.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.670+0000 7fafb037c700 1 --2- 172.21.15.6:0/1365325782 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7faf9406d660 0x7faf9406fb20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fafa80780f0 tx=0x7faf98009260 comp rx=0 tx=0).stop 2024-03-24T12:10:29.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.670+0000 7fafb037c700 1 -- 172.21.15.6:0/1365325782 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafa8069920 msgr2=0x7fafa8100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:29.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.670+0000 7fafb037c700 1 --2- 172.21.15.6:0/1365325782 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafa8069920 0x7fafa8100c40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fafa400b580 tx=0x7fafa400b9a0 comp rx=0 tx=0).stop 2024-03-24T12:10:29.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.670+0000 7fafb037c700 1 -- 172.21.15.6:0/1365325782 shutdown_connections 2024-03-24T12:10:29.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.670+0000 7fafb037c700 1 --2- 172.21.15.6:0/1365325782 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fafa8107cc0 0x7fafa81016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:29.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.670+0000 7fafb037c700 1 --2- 172.21.15.6:0/1365325782 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fafa8069920 0x7fafa8100c40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:29.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.671+0000 7fafb037c700 1 --2- 172.21.15.6:0/1365325782 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7faf9406d660 0x7faf9406fb20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:29.670 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.671+0000 7fafb037c700 1 --2- 172.21.15.6:0/1365325782 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fafa806a2c0 0x7fafa8101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:29.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.671+0000 7fafb037c700 1 -- 172.21.15.6:0/1365325782 >> 172.21.15.6:0/1365325782 conn(0x7fafa8077aa0 msgr2=0x7fafa80ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:29.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.671+0000 7fafb037c700 1 -- 172.21.15.6:0/1365325782 shutdown_connections 2024-03-24T12:10:29.671 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:29.671+0000 7fafb037c700 1 -- 172.21.15.6:0/1365325782 wait complete. 2024-03-24T12:10:29.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:29 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3971397626' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-24T12:10:30.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:30 smithi006 ceph-mon[30108]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:30.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:30 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1365325782' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-24T12:10:30.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:30 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:30.697 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:30 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:30.698 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:30 smithi006 ceph-mon[41734]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:30.698 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:30 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1365325782' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-24T12:10:30.698 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:30 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:30.698 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:30 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:30.724 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574893 got 55834574897 for osd.1 2024-03-24T12:10:30.724 DEBUG:teuthology.parallel:result is None 2024-03-24T12:10:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:30 smithi090 ceph-mon[33646]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:30 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1365325782' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-24T12:10:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:30 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:30 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:31.175 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738418 got 34359738421 for osd.0 2024-03-24T12:10:31.175 DEBUG:teuthology.parallel:result is None 2024-03-24T12:10:31.825 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:31 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:10:31.825 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:31 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:10:31.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:31 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:10:32.219 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411367 got 77309411372 for osd.2 2024-03-24T12:10:32.219 DEBUG:teuthology.parallel:result is None 2024-03-24T12:10:32.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:10:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:10:32.703 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:32 smithi006 ceph-mon[30108]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:32.703 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:32 smithi006 ceph-mon[41734]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.770+0000 7ff5af9bc700 1 -- 172.21.15.6:0/573619029 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5a8100b70 msgr2=0x7ff5a8100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.770+0000 7ff5af9bc700 1 --2- 172.21.15.6:0/573619029 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5a8100b70 0x7ff5a8100ff0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff598009990 tx=0x7ff598009ca0 comp rx=0 tx=0).stop 2024-03-24T12:10:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.770+0000 7ff5af9bc700 1 -- 172.21.15.6:0/573619029 shutdown_connections 2024-03-24T12:10:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.770+0000 7ff5af9bc700 1 --2- 172.21.15.6:0/573619029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5a8101530 0x7ff5a810d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.770+0000 7ff5af9bc700 1 --2- 172.21.15.6:0/573619029 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5a8106b80 0x7ff5a8106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.770+0000 7ff5af9bc700 1 --2- 172.21.15.6:0/573619029 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5a8100b70 0x7ff5a8100ff0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.770+0000 7ff5af9bc700 1 -- 172.21.15.6:0/573619029 >> 172.21.15.6:0/573619029 conn(0x7ff5a8077aa0 msgr2=0x7ff5a8077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.770+0000 7ff5af9bc700 1 -- 172.21.15.6:0/573619029 shutdown_connections 2024-03-24T12:10:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.770+0000 7ff5af9bc700 1 -- 172.21.15.6:0/573619029 wait complete. 2024-03-24T12:10:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.771+0000 7ff5af9bc700 1 Processor -- start 2024-03-24T12:10:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.771+0000 7ff5af9bc700 1 -- start start 2024-03-24T12:10:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.771+0000 7ff5af9bc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5a8100b70 0x7ff5a8197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.771+0000 7ff5af9bc700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5a8101530 0x7ff5a8197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.771+0000 7ff5af9bc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5a8106b80 0x7ff5a819bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.771+0000 7ff5af9bc700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a819c5e0 con 0x7ff5a8100b70 2024-03-24T12:10:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.771+0000 7ff5af9bc700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a819c750 con 0x7ff5a8106b80 2024-03-24T12:10:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.771+0000 7ff5af9bc700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a819c910 con 0x7ff5a8101530 2024-03-24T12:10:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.771+0000 7ff5ad758700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5a8100b70 0x7ff5a8197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.771+0000 7ff5acf57700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5a8101530 0x7ff5a8197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.771+0000 7ff5acf57700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5a8101530 0x7ff5a8197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:35886/0 (socket says 172.21.15.6:35886) 2024-03-24T12:10:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.771+0000 7ff5acf57700 1 -- 172.21.15.6:0/1315281867 learned_addr learned my addr 172.21.15.6:0/1315281867 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:10:32.797 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.771+0000 7ff5acf57700 1 -- 172.21.15.6:0/1315281867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5a8106b80 msgr2=0x7ff5a819bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.771+0000 7ff5adf59700 1 --2- 172.21.15.6:0/1315281867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5a8106b80 0x7ff5a819bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.771+0000 7ff5acf57700 1 --2- 172.21.15.6:0/1315281867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5a8106b80 0x7ff5a819bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.771+0000 7ff5acf57700 1 -- 172.21.15.6:0/1315281867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5a8100b70 msgr2=0x7ff5a8197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.771+0000 7ff5acf57700 1 --2- 172.21.15.6:0/1315281867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5a8100b70 0x7ff5a8197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.771+0000 7ff5acf57700 1 -- 172.21.15.6:0/1315281867 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a4009520 con 0x7ff5a8101530 2024-03-24T12:10:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.771+0000 7ff5ad758700 1 --2- 172.21.15.6:0/1315281867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5a8100b70 0x7ff5a8197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:10:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.772+0000 7ff5acf57700 1 --2- 172.21.15.6:0/1315281867 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5a8101530 0x7ff5a8197c40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff598004730 tx=0x7ff598004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.772+0000 7ff59e7fc700 1 -- 172.21.15.6:0/1315281867 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59801d070 con 0x7ff5a8101530 2024-03-24T12:10:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.772+0000 7ff59e7fc700 1 -- 172.21.15.6:0/1315281867 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ff59800be20 con 0x7ff5a8101530 2024-03-24T12:10:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.772+0000 7ff5af9bc700 1 -- 172.21.15.6:0/1315281867 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5980095f0 con 0x7ff5a8101530 2024-03-24T12:10:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.772+0000 7ff59e7fc700 1 -- 172.21.15.6:0/1315281867 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598005290 con 0x7ff5a8101530 2024-03-24T12:10:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.772+0000 7ff5af9bc700 1 -- 172.21.15.6:0/1315281867 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a819ce40 con 0x7ff5a8101530 2024-03-24T12:10:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.773+0000 7ff5af9bc700 1 -- 172.21.15.6:0/1315281867 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a8068930 con 0x7ff5a8101530 2024-03-24T12:10:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.773+0000 7ff59e7fc700 1 -- 172.21.15.6:0/1315281867 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7ff5980053f0 con 0x7ff5a8101530 2024-03-24T12:10:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.774+0000 7ff59e7fc700 1 --2- 172.21.15.6:0/1315281867 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7ff59406d660 0x7ff59406fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:32.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.774+0000 7ff59e7fc700 1 -- 172.21.15.6:0/1315281867 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7ff59808e260 con 0x7ff5a8101530 2024-03-24T12:10:32.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.774+0000 7ff5ad758700 1 --2- 172.21.15.6:0/1315281867 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7ff59406d660 0x7ff59406fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:32.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.774+0000 7ff5ad758700 1 --2- 172.21.15.6:0/1315281867 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7ff59406d660 0x7ff59406fb20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4009fd0 tx=0x7ff5a4012040 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:32.799 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.783+0000 7ff59e7fc700 1 -- 172.21.15.6:0/1315281867 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff598058700 con 0x7ff5a8101530 2024-03-24T12:10:32.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.694+0000 7f9431112700 1 -- 172.21.15.6:0/4095045372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c101560 msgr2=0x7f942c10d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:32.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.694+0000 7f9431112700 1 --2- 172.21.15.6:0/4095045372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c101560 0x7f942c10d9a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f942400b0a0 tx=0x7f942400b3b0 comp rx=0 tx=0).stop 2024-03-24T12:10:32.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.694+0000 7f9431112700 1 -- 172.21.15.6:0/4095045372 shutdown_connections 2024-03-24T12:10:32.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.694+0000 7f9431112700 1 --2- 172.21.15.6:0/4095045372 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c101560 0x7f942c10d9a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:32.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.694+0000 7f9431112700 1 --2- 172.21.15.6:0/4095045372 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f942c106bd0 0x7f942c106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:32.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.694+0000 7f9431112700 1 --2- 172.21.15.6:0/4095045372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f942c100ba0 0x7f942c101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:32.922 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.694+0000 7f9431112700 1 -- 172.21.15.6:0/4095045372 >> 172.21.15.6:0/4095045372 conn(0x7f942c076bc0 msgr2=0x7f942c076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:32.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.694+0000 7f9431112700 1 -- 172.21.15.6:0/4095045372 shutdown_connections 2024-03-24T12:10:32.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.694+0000 7f9431112700 1 -- 172.21.15.6:0/4095045372 wait complete. 2024-03-24T12:10:32.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.695+0000 7f9431112700 1 Processor -- start 2024-03-24T12:10:32.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.695+0000 7f9431112700 1 -- start start 2024-03-24T12:10:32.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.695+0000 7f9431112700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f942c100ba0 0x7f942c1963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:32.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.695+0000 7f9431112700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f942c101560 0x7f942c1968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:32.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.695+0000 7f9431112700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c106bd0 0x7f942c196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:32.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.695+0000 7f9431112700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c1a5af0 con 0x7f942c106bd0 2024-03-24T12:10:32.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.695+0000 7f9431112700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c1a5c60 con 0x7f942c101560 2024-03-24T12:10:32.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.695+0000 7f9431112700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942c1a5e20 con 0x7f942c100ba0 2024-03-24T12:10:32.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.695+0000 7f942ad9d700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f942c100ba0 0x7f942c1963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:32.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.695+0000 7f942ad9d700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f942c100ba0 0x7f942c1963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:35868/0 (socket says 172.21.15.6:35868) 2024-03-24T12:10:32.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.695+0000 7f942ad9d700 1 -- 172.21.15.6:0/3106386062 learned_addr learned my addr 172.21.15.6:0/3106386062 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:10:32.923 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.695+0000 7f942b59e700 1 --2- 172.21.15.6:0/3106386062 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c106bd0 0x7f942c196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:32.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.696+0000 7f942a59c700 1 --2- 172.21.15.6:0/3106386062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f942c101560 0x7f942c1968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:32.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.696+0000 7f942ad9d700 1 -- 172.21.15.6:0/3106386062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f942c101560 msgr2=0x7f942c1968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:32.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.696+0000 7f942ad9d700 1 --2- 172.21.15.6:0/3106386062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f942c101560 0x7f942c1968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:32.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.696+0000 7f942ad9d700 1 -- 172.21.15.6:0/3106386062 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c106bd0 msgr2=0x7f942c196e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:32.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.696+0000 7f942ad9d700 1 --2- 172.21.15.6:0/3106386062 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c106bd0 0x7f942c196e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:32.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.696+0000 7f942ad9d700 1 -- 172.21.15.6:0/3106386062 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9424009d00 con 0x7f942c100ba0 2024-03-24T12:10:32.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.696+0000 7f942b59e700 1 --2- 172.21.15.6:0/3106386062 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c106bd0 0x7f942c196e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:10:32.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.696+0000 7f942ad9d700 1 --2- 172.21.15.6:0/3106386062 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f942c100ba0 0x7f942c1963b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f941c00e9a0 tx=0x7f941c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:32.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.696+0000 7f9423fff700 1 -- 172.21.15.6:0/3106386062 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c009790 con 0x7f942c100ba0 2024-03-24T12:10:32.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.696+0000 7f9423fff700 1 -- 172.21.15.6:0/3106386062 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f941c00cbd0 con 0x7f942c100ba0 2024-03-24T12:10:32.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.696+0000 7f9431112700 1 -- 172.21.15.6:0/3106386062 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942c1a6100 con 0x7f942c100ba0 2024-03-24T12:10:32.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.696+0000 7f9423fff700 1 -- 172.21.15.6:0/3106386062 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c018420 con 0x7f942c100ba0 2024-03-24T12:10:32.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.696+0000 7f9431112700 1 -- 172.21.15.6:0/3106386062 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942c1a6600 con 0x7f942c100ba0 2024-03-24T12:10:32.924 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.697+0000 7f9431112700 1 -- 172.21.15.6:0/3106386062 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942c109060 con 0x7f942c100ba0 2024-03-24T12:10:32.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.698+0000 7f9423fff700 1 -- 172.21.15.6:0/3106386062 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7f941c010430 con 0x7f942c100ba0 2024-03-24T12:10:32.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.698+0000 7f9423fff700 1 --2- 172.21.15.6:0/3106386062 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f941806d660 0x7f941806fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:32.925 INFO:teuthology.orchestra.run.smithi006.stdout:231928234002 2024-03-24T12:10:32.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.699+0000 7f9423fff700 1 -- 172.21.15.6:0/3106386062 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f941c014070 con 0x7f942c100ba0 2024-03-24T12:10:32.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.699+0000 7f942a59c700 1 --2- 172.21.15.6:0/3106386062 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f941806d660 0x7f941806fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:32.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.699+0000 7f942a59c700 1 --2- 172.21.15.6:0/3106386062 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f941806d660 0x7f941806fb20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9414005ea0 tx=0x7f9414005e30 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:32.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.703+0000 7f9423fff700 1 -- 172.21.15.6:0/3106386062 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f941c052d20 con 0x7f942c100ba0 2024-03-24T12:10:32.925 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.917+0000 7f9431112700 1 -- 172.21.15.6:0/3106386062 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f942c06a7a0 con 0x7f942c100ba0 2024-03-24T12:10:32.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.918+0000 7f9423fff700 1 -- 172.21.15.6:0/3106386062 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f941c056b80 con 0x7f942c100ba0 2024-03-24T12:10:32.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.920+0000 7f9431112700 1 -- 172.21.15.6:0/3106386062 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f941806d660 msgr2=0x7f941806fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:32.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.920+0000 7f9431112700 1 --2- 172.21.15.6:0/3106386062 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f941806d660 0x7f941806fb20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9414005ea0 tx=0x7f9414005e30 comp rx=0 tx=0).stop 2024-03-24T12:10:32.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.920+0000 7f9431112700 1 -- 172.21.15.6:0/3106386062 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f942c100ba0 msgr2=0x7f942c1963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:32.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.920+0000 7f9431112700 1 --2- 172.21.15.6:0/3106386062 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f942c100ba0 0x7f942c1963b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f941c00e9a0 tx=0x7f941c00ecb0 comp rx=0 tx=0).stop 2024-03-24T12:10:32.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.920+0000 7f9431112700 1 -- 172.21.15.6:0/3106386062 shutdown_connections 2024-03-24T12:10:32.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.920+0000 7f9431112700 1 --2- 172.21.15.6:0/3106386062 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f942c106bd0 0x7f942c196e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:32.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.920+0000 7f9431112700 1 --2- 172.21.15.6:0/3106386062 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f941806d660 0x7f941806fb20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:32.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.920+0000 7f9431112700 1 --2- 172.21.15.6:0/3106386062 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f942c100ba0 0x7f942c1963b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:32.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.920+0000 7f9431112700 1 --2- 172.21.15.6:0/3106386062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f942c101560 0x7f942c1968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:32.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.920+0000 7f9431112700 1 -- 172.21.15.6:0/3106386062 >> 172.21.15.6:0/3106386062 conn(0x7f942c076bc0 msgr2=0x7f942c0ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:32.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.920+0000 7f9431112700 1 -- 172.21.15.6:0/3106386062 shutdown_connections 2024-03-24T12:10:32.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.920+0000 7f9431112700 1 -- 172.21.15.6:0/3106386062 wait complete. 2024-03-24T12:10:32.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:32 smithi090 ceph-mon[33646]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:32.987 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.988+0000 7ff5af9bc700 1 -- 172.21.15.6:0/1315281867 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff5a819ce40 con 0x7ff5a8101530 2024-03-24T12:10:32.988 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.989+0000 7ff59e7fc700 1 -- 172.21.15.6:0/1315281867 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff5a819ce40 con 0x7ff5a8101530 2024-03-24T12:10:32.988 INFO:teuthology.orchestra.run.smithi006.stdout:107374182439 2024-03-24T12:10:32.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.991+0000 7ff5af9bc700 1 -- 172.21.15.6:0/1315281867 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7ff59406d660 msgr2=0x7ff59406fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:32.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.991+0000 7ff5af9bc700 1 --2- 172.21.15.6:0/1315281867 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7ff59406d660 0x7ff59406fb20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4009fd0 tx=0x7ff5a4012040 comp rx=0 tx=0).stop 2024-03-24T12:10:32.990 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.991+0000 7ff5af9bc700 1 -- 172.21.15.6:0/1315281867 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5a8101530 msgr2=0x7ff5a8197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:32.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.991+0000 7ff5af9bc700 1 --2- 172.21.15.6:0/1315281867 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5a8101530 0x7ff5a8197c40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff598004730 tx=0x7ff598004760 comp rx=0 tx=0).stop 2024-03-24T12:10:32.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.991+0000 7ff5af9bc700 1 -- 172.21.15.6:0/1315281867 shutdown_connections 2024-03-24T12:10:32.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.991+0000 7ff5af9bc700 1 --2- 172.21.15.6:0/1315281867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5a8106b80 0x7ff5a819bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:32.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.991+0000 7ff5af9bc700 1 --2- 172.21.15.6:0/1315281867 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5a8100b70 0x7ff5a8197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:32.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.991+0000 7ff5af9bc700 1 --2- 172.21.15.6:0/1315281867 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7ff59406d660 0x7ff59406fb20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:32.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.991+0000 7ff5af9bc700 1 --2- 172.21.15.6:0/1315281867 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5a8101530 0x7ff5a8197c40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:32.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.991+0000 7ff5af9bc700 1 -- 172.21.15.6:0/1315281867 >> 172.21.15.6:0/1315281867 conn(0x7ff5a8077aa0 msgr2=0x7ff5a80ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:32.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.991+0000 7ff5af9bc700 1 -- 172.21.15.6:0/1315281867 shutdown_connections 2024-03-24T12:10:32.991 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:32.991+0000 7ff5af9bc700 1 -- 172.21.15.6:0/1315281867 wait complete. 2024-03-24T12:10:33.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:33 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:10:33.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3106386062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-24T12:10:33.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1315281867' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-24T12:10:33.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:33 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:33.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:33 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:33.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:33 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:33 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:33 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:33 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:33 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:33 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:33 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:33 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:33 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:33.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:33 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:10:33.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:33 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:10:33.940 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:10:33.940 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3106386062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-24T12:10:33.940 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1315281867' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-24T12:10:33.940 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:33.940 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:33.940 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.940 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.940 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.940 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.940 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.940 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.941 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.941 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.941 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:33.941 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:10:33.941 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:10:33.941 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:10:33.941 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3106386062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-24T12:10:33.941 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1315281867' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-24T12:10:33.941 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:33.941 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:33.941 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.941 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.941 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.942 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.942 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.942 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.942 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.942 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:10:33.942 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:33.942 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:10:33.942 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:33 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:10:33.968 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182436 got 107374182439 for osd.3 2024-03-24T12:10:33.969 DEBUG:teuthology.parallel:result is None 2024-03-24T12:10:34.421 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928233999 got 231928234002 for osd.7 2024-03-24T12:10:34.421 DEBUG:teuthology.parallel:result is None 2024-03-24T12:10:34.421 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-24T12:10:34.421 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-24T12:10:34.674 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:10:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:34 smithi006 ceph-mon[30108]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:10:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:34 smithi006 ceph-mon[30108]: Adjusting osd_memory_target on smithi006 to 3761M 2024-03-24T12:10:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:34 smithi006 ceph-mon[30108]: Updating smithi006:/etc/ceph/ceph.conf 2024-03-24T12:10:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:34 smithi006 ceph-mon[30108]: Updating smithi090:/etc/ceph/ceph.conf 2024-03-24T12:10:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:34 smithi006 ceph-mon[30108]: Updating smithi006:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T12:10:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:34 smithi006 ceph-mon[30108]: Updating smithi090:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T12:10:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:34 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:34 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:34 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:34 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:34 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:34.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:34 smithi006 ceph-mon[41734]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:10:34.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:34 smithi006 ceph-mon[41734]: Adjusting osd_memory_target on smithi006 to 3761M 2024-03-24T12:10:34.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:34 smithi006 ceph-mon[41734]: Updating smithi006:/etc/ceph/ceph.conf 2024-03-24T12:10:34.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:34 smithi006 ceph-mon[41734]: Updating smithi090:/etc/ceph/ceph.conf 2024-03-24T12:10:34.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:34 smithi006 ceph-mon[41734]: Updating smithi006:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T12:10:34.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:34 smithi006 ceph-mon[41734]: Updating smithi090:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T12:10:34.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:34 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:34.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:34 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:34.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:34 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:34.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:34 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:34.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:34 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:34.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:34 smithi090 ceph-mon[33646]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:10:34.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:34 smithi090 ceph-mon[33646]: Adjusting osd_memory_target on smithi006 to 3761M 2024-03-24T12:10:34.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:34 smithi090 ceph-mon[33646]: Updating smithi006:/etc/ceph/ceph.conf 2024-03-24T12:10:34.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:34 smithi090 ceph-mon[33646]: Updating smithi090:/etc/ceph/ceph.conf 2024-03-24T12:10:34.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:34 smithi090 ceph-mon[33646]: Updating smithi006:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T12:10:34.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:34 smithi090 ceph-mon[33646]: Updating smithi090:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-24T12:10:34.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:34 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:34.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:34 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:34.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:34 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:34.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:34 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:34.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:34 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:35.461 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:10:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:10:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:10:35.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:35 smithi090 ceph-mon[33646]: Updating smithi006:/etc/ceph/ceph.client.admin.keyring 2024-03-24T12:10:35.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:35 smithi090 ceph-mon[33646]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-03-24T12:10:35.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:35 smithi090 ceph-mon[33646]: Updating smithi006:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T12:10:35.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:35 smithi090 ceph-mon[33646]: Updating smithi090:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T12:10:35.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:35 smithi090 ceph-mon[33646]: Deploying daemon alertmanager.a on smithi006 2024-03-24T12:10:35.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:35 smithi006 ceph-mon[30108]: Updating smithi006:/etc/ceph/ceph.client.admin.keyring 2024-03-24T12:10:35.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:35 smithi006 ceph-mon[30108]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-03-24T12:10:35.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:35 smithi006 ceph-mon[30108]: Updating smithi006:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T12:10:35.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:35 smithi006 ceph-mon[30108]: Updating smithi090:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T12:10:35.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:35 smithi006 ceph-mon[30108]: Deploying daemon alertmanager.a on smithi006 2024-03-24T12:10:35.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:35 smithi006 ceph-mon[41734]: Updating smithi006:/etc/ceph/ceph.client.admin.keyring 2024-03-24T12:10:35.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:35 smithi006 ceph-mon[41734]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-03-24T12:10:35.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:35 smithi006 ceph-mon[41734]: Updating smithi006:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T12:10:35.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:35 smithi006 ceph-mon[41734]: Updating smithi090:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-24T12:10:35.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:35 smithi006 ceph-mon[41734]: Deploying daemon alertmanager.a on smithi006 2024-03-24T12:10:36.736 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:36 smithi006 ceph-mon[30108]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:36.737 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:36 smithi006 ceph-mon[41734]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:36.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.848+0000 7fe8e10b5700 1 -- 172.21.15.6:0/2265665598 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe8dc106ff0 msgr2=0x7fe8dc10eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:36.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.848+0000 7fe8e10b5700 1 --2- 172.21.15.6:0/2265665598 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe8dc106ff0 0x7fe8dc10eb70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400b0a0 tx=0x7fe8d400b3b0 comp rx=0 tx=0).stop 2024-03-24T12:10:36.896 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.849+0000 7fe8e10b5700 1 -- 172.21.15.6:0/2265665598 shutdown_connections 2024-03-24T12:10:36.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.849+0000 7fe8e10b5700 1 --2- 172.21.15.6:0/2265665598 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe8dc106ff0 0x7fe8dc10eb70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:36.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.849+0000 7fe8e10b5700 1 --2- 172.21.15.6:0/2265665598 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8dc06a410 0x7fe8dc06a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:36.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.849+0000 7fe8e10b5700 1 --2- 172.21.15.6:0/2265665598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe8dc06adc0 0x7fe8dc1068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:36.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.849+0000 7fe8e10b5700 1 -- 172.21.15.6:0/2265665598 >> 172.21.15.6:0/2265665598 conn(0x7fe8dc076bc0 msgr2=0x7fe8dc076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:36.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.849+0000 7fe8e10b5700 1 -- 172.21.15.6:0/2265665598 shutdown_connections 2024-03-24T12:10:36.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.849+0000 7fe8e10b5700 1 -- 172.21.15.6:0/2265665598 wait complete. 2024-03-24T12:10:36.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.849+0000 7fe8e10b5700 1 Processor -- start 2024-03-24T12:10:36.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.850+0000 7fe8e10b5700 1 -- start start 2024-03-24T12:10:36.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.850+0000 7fe8e10b5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe8dc06a410 0x7fe8dc19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:36.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.850+0000 7fe8e10b5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8dc06adc0 0x7fe8dc19d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:36.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.850+0000 7fe8e10b5700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe8dc106ff0 0x7fe8dc197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:36.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.850+0000 7fe8e10b5700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc197c70 con 0x7fe8dc06adc0 2024-03-24T12:10:36.897 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.850+0000 7fe8e10b5700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc197db0 con 0x7fe8dc06a410 2024-03-24T12:10:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.850+0000 7fe8e10b5700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc197f70 con 0x7fe8dc106ff0 2024-03-24T12:10:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.850+0000 7fe8da59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8dc06adc0 0x7fe8dc19d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.850+0000 7fe8db59e700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe8dc106ff0 0x7fe8dc197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.850+0000 7fe8da59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8dc06adc0 0x7fe8dc19d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:42388/0 (socket says 172.21.15.6:42388) 2024-03-24T12:10:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.850+0000 7fe8da59c700 1 -- 172.21.15.6:0/4023859040 learned_addr learned my addr 172.21.15.6:0/4023859040 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:10:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.850+0000 7fe8dad9d700 1 --2- 172.21.15.6:0/4023859040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe8dc06a410 0x7fe8dc19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.850+0000 7fe8db59e700 1 -- 172.21.15.6:0/4023859040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe8dc06a410 msgr2=0x7fe8dc19d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.850+0000 7fe8db59e700 1 --2- 172.21.15.6:0/4023859040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe8dc06a410 0x7fe8dc19d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.850+0000 7fe8db59e700 1 -- 172.21.15.6:0/4023859040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8dc06adc0 msgr2=0x7fe8dc19d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.850+0000 7fe8db59e700 1 --2- 172.21.15.6:0/4023859040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8dc06adc0 0x7fe8dc19d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.850+0000 7fe8db59e700 1 -- 172.21.15.6:0/4023859040 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8cc009520 con 0x7fe8dc106ff0 2024-03-24T12:10:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.850+0000 7fe8da59c700 1 --2- 172.21.15.6:0/4023859040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8dc06adc0 0x7fe8dc19d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:10:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.851+0000 7fe8db59e700 1 --2- 172.21.15.6:0/4023859040 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe8dc106ff0 0x7fe8dc197610 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400b070 tx=0x7fe8d4007930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.851+0000 7fe8c3fff700 1 -- 172.21.15.6:0/4023859040 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d400e040 con 0x7fe8dc106ff0 2024-03-24T12:10:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.851+0000 7fe8e10b5700 1 -- 172.21.15.6:0/4023859040 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d4009d00 con 0x7fe8dc106ff0 2024-03-24T12:10:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.851+0000 7fe8c3fff700 1 -- 172.21.15.6:0/4023859040 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe8d4004c50 con 0x7fe8dc106ff0 2024-03-24T12:10:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.851+0000 7fe8c3fff700 1 -- 172.21.15.6:0/4023859040 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4005040 con 0x7fe8dc106ff0 2024-03-24T12:10:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.851+0000 7fe8e10b5700 1 -- 172.21.15.6:0/4023859040 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8dc1a4320 con 0x7fe8dc106ff0 2024-03-24T12:10:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.852+0000 7fe8e10b5700 1 -- 172.21.15.6:0/4023859040 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8dc10a1e0 con 0x7fe8dc106ff0 2024-03-24T12:10:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.852+0000 7fe8c3fff700 1 -- 172.21.15.6:0/4023859040 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7fe8d401d470 con 0x7fe8dc106ff0 2024-03-24T12:10:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.853+0000 7fe8c3fff700 1 --2- 172.21.15.6:0/4023859040 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fe8c406d660 0x7fe8c406fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.853+0000 7fe8c3fff700 1 -- 172.21.15.6:0/4023859040 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7fe8d408e7b0 con 0x7fe8dc106ff0 2024-03-24T12:10:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.853+0000 7fe8dad9d700 1 --2- 172.21.15.6:0/4023859040 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fe8c406d660 0x7fe8c406fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.853+0000 7fe8dad9d700 1 --2- 172.21.15.6:0/4023859040 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fe8c406d660 0x7fe8c406fb20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc009d80 tx=0x7fe8cc009260 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:36.858+0000 7fe8c3fff700 1 -- 172.21.15.6:0/4023859040 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe8d4057e00 con 0x7fe8dc106ff0 2024-03-24T12:10:36.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:36 smithi090 ceph-mon[33646]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:37.091 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:37.091+0000 7fe8e10b5700 1 -- 172.21.15.6:0/4023859040 --> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe8dc068930 con 0x7fe8c406d660 2024-03-24T12:10:37.103 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:37.104+0000 7fe8c3fff700 1 -- 172.21.15.6:0/4023859040 <== mgr.24464 v2:172.21.15.6:6800/3865852080 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+346476 (secure 0 0 0) 0x7fe8dc068930 con 0x7fe8c406d660 2024-03-24T12:10:37.108 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:10:37.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:37.108+0000 7fe8e10b5700 1 -- 172.21.15.6:0/4023859040 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fe8c406d660 msgr2=0x7fe8c406fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:37.108+0000 7fe8e10b5700 1 --2- 172.21.15.6:0/4023859040 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fe8c406d660 0x7fe8c406fb20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc009d80 tx=0x7fe8cc009260 comp rx=0 tx=0).stop 2024-03-24T12:10:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:37.108+0000 7fe8e10b5700 1 -- 172.21.15.6:0/4023859040 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe8dc106ff0 msgr2=0x7fe8dc197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:37.108+0000 7fe8e10b5700 1 --2- 172.21.15.6:0/4023859040 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe8dc106ff0 0x7fe8dc197610 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400b070 tx=0x7fe8d4007930 comp rx=0 tx=0).stop 2024-03-24T12:10:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:37.108+0000 7fe8e10b5700 1 -- 172.21.15.6:0/4023859040 shutdown_connections 2024-03-24T12:10:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:37.108+0000 7fe8e10b5700 1 --2- 172.21.15.6:0/4023859040 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fe8c406d660 0x7fe8c406fb20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:37.108+0000 7fe8e10b5700 1 --2- 172.21.15.6:0/4023859040 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe8dc106ff0 0x7fe8dc197610 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:37.108+0000 7fe8e10b5700 1 --2- 172.21.15.6:0/4023859040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe8dc06a410 0x7fe8dc19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:37.108+0000 7fe8e10b5700 1 --2- 172.21.15.6:0/4023859040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8dc06adc0 0x7fe8dc19d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:37.108+0000 7fe8e10b5700 1 -- 172.21.15.6:0/4023859040 >> 172.21.15.6:0/4023859040 conn(0x7fe8dc076bc0 msgr2=0x7fe8dc0fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:37.109+0000 7fe8e10b5700 1 -- 172.21.15.6:0/4023859040 shutdown_connections 2024-03-24T12:10:37.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:37.109+0000 7fe8e10b5700 1 -- 172.21.15.6:0/4023859040 wait complete. 2024-03-24T12:10:37.112 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-03-24T12:10:38.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:38 smithi090 ceph-mon[33646]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:10:38.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:38 smithi090 ceph-mon[33646]: from='client.24602 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T12:10:38.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:38 smithi006 ceph-mon[30108]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:10:38.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:38 smithi006 ceph-mon[30108]: from='client.24602 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T12:10:38.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:38 smithi006 ceph-mon[41734]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:10:38.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:38 smithi006 ceph-mon[41734]: from='client.24602 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T12:10:39.078 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-03-24T12:10:37.012166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596318,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1003,"num_read_kb":849,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1771434,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":245040,"kb_used_data":8092,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":749716176,"statfs":{"total":767960285184,"available":767709364224,"internally_reserved":0,"allocated":8286208,"data_stored":4480994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003041"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997070+0000","last_change":"2024-03-24T12:09:27.111366+0000","last_active":"2024-03-24T12:10:00.997070+0000","last_peered":"2024-03-24T12:10:00.997070+0000","last_clean":"2024-03-24T12:10:00.997070+0000","last_became_active":"2024-03-24T12:09:27.110948+0000","last_became_peered":"2024-03-24T12:09:27.110948+0000","last_unstale":"2024-03-24T12:10:00.997070+0000","last_undegraded":"2024-03-24T12:10:00.997070+0000","last_fullsized":"2024-03-24T12:10:00.997070+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:35:32.019484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317036+0000","last_change":"2024-03-24T12:09:25.100258+0000","last_active":"2024-03-24T12:10:01.317036+0000","last_peered":"2024-03-24T12:10:01.317036+0000","last_clean":"2024-03-24T12:10:01.317036+0000","last_became_active":"2024-03-24T12:09:25.099947+0000","last_became_peered":"2024-03-24T12:09:25.099947+0000","last_unstale":"2024-03-24T12:10:01.317036+0000","last_undegraded":"2024-03-24T12:10:01.317036+0000","last_fullsized":"2024-03-24T12:10:01.317036+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:30:16.001334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.995075+0000","last_change":"2024-03-24T12:09:21.445304+0000","last_active":"2024-03-24T12:10:00.995075+0000","last_peered":"2024-03-24T12:10:00.995075+0000","last_clean":"2024-03-24T12:10:00.995075+0000","last_became_active":"2024-03-24T12:09:21.444950+0000","last_became_peered":"2024-03-24T12:09:21.444950+0000","last_unstale":"2024-03-24T12:10:00.995075+0000","last_undegraded":"2024-03-24T12:10:00.995075+0000","last_fullsized":"2024-03-24T12:10:00.995075+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T15:05:05.674601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.996981+0000","last_change":"2024-03-24T12:09:23.077223+0000","last_active":"2024-03-24T12:10:00.996981+0000","last_peered":"2024-03-24T12:10:00.996981+0000","last_clean":"2024-03-24T12:10:00.996981+0000","last_became_active":"2024-03-24T12:09:23.076849+0000","last_became_peered":"2024-03-24T12:09:23.076849+0000","last_unstale":"2024-03-24T12:10:00.996981+0000","last_undegraded":"2024-03-24T12:10:00.996981+0000","last_fullsized":"2024-03-24T12:10:00.996981+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T23:43:37.825165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"68'1","reported_seq":22,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317233+0000","last_change":"2024-03-24T12:09:27.150853+0000","last_active":"2024-03-24T12:10:01.317233+0000","last_peered":"2024-03-24T12:10:01.317233+0000","last_clean":"2024-03-24T12:10:01.317233+0000","last_became_active":"2024-03-24T12:09:27.150496+0000","last_became_peered":"2024-03-24T12:09:27.150496+0000","last_unstale":"2024-03-24T12:10:01.317233+0000","last_undegraded":"2024-03-24T12:10:01.317233+0000","last_fullsized":"2024-03-24T12:10:01.317233+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T23:37:35.131084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.996988+0000","last_change":"2024-03-24T12:09:25.102274+0000","last_active":"2024-03-24T12:10:00.996988+0000","last_peered":"2024-03-24T12:10:00.996988+0000","last_clean":"2024-03-24T12:10:00.996988+0000","last_became_active":"2024-03-24T12:09:25.101995+0000","last_became_peered":"2024-03-24T12:09:25.101995+0000","last_unstale":"2024-03-24T12:10:00.996988+0000","last_undegraded":"2024-03-24T12:10:00.996988+0000","last_fullsized":"2024-03-24T12:10:00.996988+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:19:07.281270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.318722+0000","last_change":"2024-03-24T12:09:21.444675+0000","last_active":"2024-03-24T12:10:01.318722+0000","last_peered":"2024-03-24T12:10:01.318722+0000","last_clean":"2024-03-24T12:10:01.318722+0000","last_became_active":"2024-03-24T12:09:21.444335+0000","last_became_peered":"2024-03-24T12:09:21.444335+0000","last_unstale":"2024-03-24T12:10:01.318722+0000","last_undegraded":"2024-03-24T12:10:01.318722+0000","last_fullsized":"2024-03-24T12:10:01.318722+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:22:11.203735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"68'15","reported_seq":54,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.319019+0000","last_change":"2024-03-24T12:09:23.086498+0000","last_active":"2024-03-24T12:10:01.319019+0000","last_peered":"2024-03-24T12:10:01.319019+0000","last_clean":"2024-03-24T12:10:01.319019+0000","last_became_active":"2024-03-24T12:09:23.086077+0000","last_became_peered":"2024-03-24T12:09:23.086077+0000","last_unstale":"2024-03-24T12:10:01.319019+0000","last_undegraded":"2024-03-24T12:10:01.319019+0000","last_fullsized":"2024-03-24T12:10:01.319019+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T22:52:20.448715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.994198+0000","last_change":"2024-03-24T12:09:27.120441+0000","last_active":"2024-03-24T12:10:00.994198+0000","last_peered":"2024-03-24T12:10:00.994198+0000","last_clean":"2024-03-24T12:10:00.994198+0000","last_became_active":"2024-03-24T12:09:27.120132+0000","last_became_peered":"2024-03-24T12:09:27.120132+0000","last_unstale":"2024-03-24T12:10:00.994198+0000","last_undegraded":"2024-03-24T12:10:00.994198+0000","last_fullsized":"2024-03-24T12:10:00.994198+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:16:03.656763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107490+0000","last_change":"2024-03-24T12:09:25.106706+0000","last_active":"2024-03-24T12:10:01.107490+0000","last_peered":"2024-03-24T12:10:01.107490+0000","last_clean":"2024-03-24T12:10:01.107490+0000","last_became_active":"2024-03-24T12:09:25.106225+0000","last_became_peered":"2024-03-24T12:09:25.106225+0000","last_unstale":"2024-03-24T12:10:01.107490+0000","last_undegraded":"2024-03-24T12:10:01.107490+0000","last_fullsized":"2024-03-24T12:10:01.107490+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:11:02.213531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107911+0000","last_change":"2024-03-24T12:09:20.824379+0000","last_active":"2024-03-24T12:10:01.107911+0000","last_peered":"2024-03-24T12:10:01.107911+0000","last_clean":"2024-03-24T12:10:01.107911+0000","last_became_active":"2024-03-24T12:09:20.823417+0000","last_became_peered":"2024-03-24T12:09:20.823417+0000","last_unstale":"2024-03-24T12:10:01.107911+0000","last_undegraded":"2024-03-24T12:10:01.107911+0000","last_fullsized":"2024-03-24T12:10:01.107911+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:35:05.153460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997256+0000","last_change":"2024-03-24T12:09:23.084076+0000","last_active":"2024-03-24T12:10:00.997256+0000","last_peered":"2024-03-24T12:10:00.997256+0000","last_clean":"2024-03-24T12:10:00.997256+0000","last_became_active":"2024-03-24T12:09:23.083717+0000","last_became_peered":"2024-03-24T12:09:23.083717+0000","last_unstale":"2024-03-24T12:10:00.997256+0000","last_undegraded":"2024-03-24T12:10:00.997256+0000","last_fullsized":"2024-03-24T12:10:00.997256+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:13:19.903943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107309+0000","last_change":"2024-03-24T12:09:27.118024+0000","last_active":"2024-03-24T12:10:01.107309+0000","last_peered":"2024-03-24T12:10:01.107309+0000","last_clean":"2024-03-24T12:10:01.107309+0000","last_became_active":"2024-03-24T12:09:27.117712+0000","last_became_peered":"2024-03-24T12:09:27.117712+0000","last_unstale":"2024-03-24T12:10:01.107309+0000","last_undegraded":"2024-03-24T12:10:01.107309+0000","last_fullsized":"2024-03-24T12:10:01.107309+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:42:22.383046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.115936+0000","last_change":"2024-03-24T12:09:25.105091+0000","last_active":"2024-03-24T12:10:01.115936+0000","last_peered":"2024-03-24T12:10:01.115936+0000","last_clean":"2024-03-24T12:10:01.115936+0000","last_became_active":"2024-03-24T12:09:25.104771+0000","last_became_peered":"2024-03-24T12:09:25.104771+0000","last_unstale":"2024-03-24T12:10:01.115936+0000","last_undegraded":"2024-03-24T12:10:01.115936+0000","last_fullsized":"2024-03-24T12:10:01.115936+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:27:28.765558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107558+0000","last_change":"2024-03-24T12:09:20.821356+0000","last_active":"2024-03-24T12:10:01.107558+0000","last_peered":"2024-03-24T12:10:01.107558+0000","last_clean":"2024-03-24T12:10:01.107558+0000","last_became_active":"2024-03-24T12:09:20.821052+0000","last_became_peered":"2024-03-24T12:09:20.821052+0000","last_unstale":"2024-03-24T12:10:01.107558+0000","last_undegraded":"2024-03-24T12:10:01.107558+0000","last_fullsized":"2024-03-24T12:10:01.107558+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:18:27.681064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"68'5","reported_seq":39,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997854+0000","last_change":"2024-03-24T12:09:23.084755+0000","last_active":"2024-03-24T12:10:00.997854+0000","last_peered":"2024-03-24T12:10:00.997854+0000","last_clean":"2024-03-24T12:10:00.997854+0000","last_became_active":"2024-03-24T12:09:23.084425+0000","last_became_peered":"2024-03-24T12:09:23.084425+0000","last_unstale":"2024-03-24T12:10:00.997854+0000","last_undegraded":"2024-03-24T12:10:00.997854+0000","last_fullsized":"2024-03-24T12:10:00.997854+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:14:43.375057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.999291+0000","last_change":"2024-03-24T12:09:27.151256+0000","last_active":"2024-03-24T12:10:00.999291+0000","last_peered":"2024-03-24T12:10:00.999291+0000","last_clean":"2024-03-24T12:10:00.999291+0000","last_became_active":"2024-03-24T12:09:27.150904+0000","last_became_peered":"2024-03-24T12:09:27.150904+0000","last_unstale":"2024-03-24T12:10:00.999291+0000","last_undegraded":"2024-03-24T12:10:00.999291+0000","last_fullsized":"2024-03-24T12:10:00.999291+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:16:44.043383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317777+0000","last_change":"2024-03-24T12:09:25.099346+0000","last_active":"2024-03-24T12:10:01.317777+0000","last_peered":"2024-03-24T12:10:01.317777+0000","last_clean":"2024-03-24T12:10:01.317777+0000","last_became_active":"2024-03-24T12:09:25.099032+0000","last_became_peered":"2024-03-24T12:09:25.099032+0000","last_unstale":"2024-03-24T12:10:01.317777+0000","last_undegraded":"2024-03-24T12:10:01.317777+0000","last_fullsized":"2024-03-24T12:10:01.317777+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:49:13.644904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.995223+0000","last_change":"2024-03-24T12:09:20.827565+0000","last_active":"2024-03-24T12:10:00.995223+0000","last_peered":"2024-03-24T12:10:00.995223+0000","last_clean":"2024-03-24T12:10:00.995223+0000","last_became_active":"2024-03-24T12:09:20.827251+0000","last_became_peered":"2024-03-24T12:09:20.827251+0000","last_unstale":"2024-03-24T12:10:00.995223+0000","last_undegraded":"2024-03-24T12:10:00.995223+0000","last_fullsized":"2024-03-24T12:10:00.995223+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:15:57.815587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"68'15","reported_seq":54,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.113419+0000","last_change":"2024-03-24T12:09:23.088416+0000","last_active":"2024-03-24T12:10:01.113419+0000","last_peered":"2024-03-24T12:10:01.113419+0000","last_clean":"2024-03-24T12:10:01.113419+0000","last_became_active":"2024-03-24T12:09:23.088067+0000","last_became_peered":"2024-03-24T12:09:23.088067+0000","last_unstale":"2024-03-24T12:10:01.113419+0000","last_undegraded":"2024-03-24T12:10:01.113419+0000","last_fullsized":"2024-03-24T12:10:01.113419+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T15:03:27.932435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.318081+0000","last_change":"2024-03-24T12:09:27.150904+0000","last_active":"2024-03-24T12:10:01.318081+0000","last_peered":"2024-03-24T12:10:01.318081+0000","last_clean":"2024-03-24T12:10:01.318081+0000","last_became_active":"2024-03-24T12:09:27.150486+0000","last_became_peered":"2024-03-24T12:09:27.150486+0000","last_unstale":"2024-03-24T12:10:01.318081+0000","last_undegraded":"2024-03-24T12:10:01.318081+0000","last_fullsized":"2024-03-24T12:10:01.318081+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T23:30:35.024129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997732+0000","last_change":"2024-03-24T12:09:25.098180+0000","last_active":"2024-03-24T12:10:00.997732+0000","last_peered":"2024-03-24T12:10:00.997732+0000","last_clean":"2024-03-24T12:10:00.997732+0000","last_became_active":"2024-03-24T12:09:25.097874+0000","last_became_peered":"2024-03-24T12:09:25.097874+0000","last_unstale":"2024-03-24T12:10:00.997732+0000","last_undegraded":"2024-03-24T12:10:00.997732+0000","last_fullsized":"2024-03-24T12:10:00.997732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:03:12.128447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997409+0000","last_change":"2024-03-24T12:09:21.444812+0000","last_active":"2024-03-24T12:10:00.997409+0000","last_peered":"2024-03-24T12:10:00.997409+0000","last_clean":"2024-03-24T12:10:00.997409+0000","last_became_active":"2024-03-24T12:09:21.444497+0000","last_became_peered":"2024-03-24T12:09:21.444497+0000","last_unstale":"2024-03-24T12:10:00.997409+0000","last_undegraded":"2024-03-24T12:10:00.997409+0000","last_fullsized":"2024-03-24T12:10:00.997409+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:50:59.952567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"68'12","reported_seq":52,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317798+0000","last_change":"2024-03-24T12:09:23.085451+0000","last_active":"2024-03-24T12:10:01.317798+0000","last_peered":"2024-03-24T12:10:01.317798+0000","last_clean":"2024-03-24T12:10:01.317798+0000","last_became_active":"2024-03-24T12:09:23.085117+0000","last_became_peered":"2024-03-24T12:09:23.085117+0000","last_unstale":"2024-03-24T12:10:01.317798+0000","last_undegraded":"2024-03-24T12:10:01.317798+0000","last_fullsized":"2024-03-24T12:10:01.317798+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:41:28.199959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"68'1","reported_seq":22,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.116290+0000","last_change":"2024-03-24T12:09:27.114268+0000","last_active":"2024-03-24T12:10:01.116290+0000","last_peered":"2024-03-24T12:10:01.116290+0000","last_clean":"2024-03-24T12:10:01.116290+0000","last_became_active":"2024-03-24T12:09:27.114007+0000","last_became_peered":"2024-03-24T12:09:27.114007+0000","last_unstale":"2024-03-24T12:10:01.116290+0000","last_undegraded":"2024-03-24T12:10:01.116290+0000","last_fullsized":"2024-03-24T12:10:01.116290+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:00:20.199850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":370,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"68'8","reported_seq":52,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:31.500733+0000","last_change":"2024-03-24T12:09:25.095318+0000","last_active":"2024-03-24T12:10:31.500733+0000","last_peered":"2024-03-24T12:10:31.500733+0000","last_clean":"2024-03-24T12:10:31.500733+0000","last_became_active":"2024-03-24T12:09:25.093987+0000","last_became_peered":"2024-03-24T12:09:25.093987+0000","last_unstale":"2024-03-24T12:10:31.500733+0000","last_undegraded":"2024-03-24T12:10:31.500733+0000","last_fullsized":"2024-03-24T12:10:31.500733+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:33:56.774618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"61'1","reported_seq":34,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317132+0000","last_change":"2024-03-24T12:09:20.820519+0000","last_active":"2024-03-24T12:10:01.317132+0000","last_peered":"2024-03-24T12:10:01.317132+0000","last_clean":"2024-03-24T12:10:01.317132+0000","last_became_active":"2024-03-24T12:09:20.820159+0000","last_became_peered":"2024-03-24T12:09:20.820159+0000","last_unstale":"2024-03-24T12:10:01.317132+0000","last_undegraded":"2024-03-24T12:10:01.317132+0000","last_fullsized":"2024-03-24T12:10:01.317132+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T22:47:19.125795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"68'10","reported_seq":44,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.994921+0000","last_change":"2024-03-24T12:09:23.145251+0000","last_active":"2024-03-24T12:10:00.994921+0000","last_peered":"2024-03-24T12:10:00.994921+0000","last_clean":"2024-03-24T12:10:00.994921+0000","last_became_active":"2024-03-24T12:09:23.144937+0000","last_became_peered":"2024-03-24T12:09:23.144937+0000","last_unstale":"2024-03-24T12:10:00.994921+0000","last_undegraded":"2024-03-24T12:10:00.994921+0000","last_fullsized":"2024-03-24T12:10:00.994921+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-26T00:05:00.172163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317487+0000","last_change":"2024-03-24T12:09:27.109797+0000","last_active":"2024-03-24T12:10:01.317487+0000","last_peered":"2024-03-24T12:10:01.317487+0000","last_clean":"2024-03-24T12:10:01.317487+0000","last_became_active":"2024-03-24T12:09:27.109445+0000","last_became_peered":"2024-03-24T12:09:27.109445+0000","last_unstale":"2024-03-24T12:10:01.317487+0000","last_undegraded":"2024-03-24T12:10:01.317487+0000","last_fullsized":"2024-03-24T12:10:01.317487+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:28:47.200374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.995756+0000","last_change":"2024-03-24T12:09:25.095275+0000","last_active":"2024-03-24T12:10:00.995756+0000","last_peered":"2024-03-24T12:10:00.995756+0000","last_clean":"2024-03-24T12:10:00.995756+0000","last_became_active":"2024-03-24T12:09:25.094721+0000","last_became_peered":"2024-03-24T12:09:25.094721+0000","last_unstale":"2024-03-24T12:10:00.995756+0000","last_undegraded":"2024-03-24T12:10:00.995756+0000","last_fullsized":"2024-03-24T12:10:00.995756+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:11:59.980171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317420+0000","last_change":"2024-03-24T12:09:20.823426+0000","last_active":"2024-03-24T12:10:01.317420+0000","last_peered":"2024-03-24T12:10:01.317420+0000","last_clean":"2024-03-24T12:10:01.317420+0000","last_became_active":"2024-03-24T12:09:20.823110+0000","last_became_peered":"2024-03-24T12:09:20.823110+0000","last_unstale":"2024-03-24T12:10:01.317420+0000","last_undegraded":"2024-03-24T12:10:01.317420+0000","last_fullsized":"2024-03-24T12:10:01.317420+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T20:12:28.514808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"68'11","reported_seq":66,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317220+0000","last_change":"2024-03-24T12:09:46.976121+0000","last_active":"2024-03-24T12:10:01.317220+0000","last_peered":"2024-03-24T12:10:01.317220+0000","last_clean":"2024-03-24T12:10:01.317220+0000","last_became_active":"2024-03-24T12:09:46.963045+0000","last_became_peered":"2024-03-24T12:09:46.963045+0000","last_unstale":"2024-03-24T12:10:01.317220+0000","last_undegraded":"2024-03-24T12:10:01.317220+0000","last_fullsized":"2024-03-24T12:10:01.317220+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:39:07.103927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.115933+0000","last_change":"2024-03-24T12:09:27.111424+0000","last_active":"2024-03-24T12:10:01.115933+0000","last_peered":"2024-03-24T12:10:01.115933+0000","last_clean":"2024-03-24T12:10:01.115933+0000","last_became_active":"2024-03-24T12:09:27.111137+0000","last_became_peered":"2024-03-24T12:09:27.111137+0000","last_unstale":"2024-03-24T12:10:01.115933+0000","last_undegraded":"2024-03-24T12:10:01.115933+0000","last_fullsized":"2024-03-24T12:10:01.115933+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:23:12.613402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.316698+0000","last_change":"2024-03-24T12:09:25.094396+0000","last_active":"2024-03-24T12:10:01.316698+0000","last_peered":"2024-03-24T12:10:01.316698+0000","last_clean":"2024-03-24T12:10:01.316698+0000","last_became_active":"2024-03-24T12:09:25.094125+0000","last_became_peered":"2024-03-24T12:09:25.094125+0000","last_unstale":"2024-03-24T12:10:01.316698+0000","last_undegraded":"2024-03-24T12:10:01.316698+0000","last_fullsized":"2024-03-24T12:10:01.316698+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T22:59:17.597978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.995239+0000","last_change":"2024-03-24T12:09:20.825947+0000","last_active":"2024-03-24T12:10:00.995239+0000","last_peered":"2024-03-24T12:10:00.995239+0000","last_clean":"2024-03-24T12:10:00.995239+0000","last_became_active":"2024-03-24T12:09:20.825525+0000","last_became_peered":"2024-03-24T12:09:20.825525+0000","last_unstale":"2024-03-24T12:10:00.995239+0000","last_undegraded":"2024-03-24T12:10:00.995239+0000","last_fullsized":"2024-03-24T12:10:00.995239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T15:29:58.677145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"68'4","reported_seq":35,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317513+0000","last_change":"2024-03-24T12:09:23.084690+0000","last_active":"2024-03-24T12:10:01.317513+0000","last_peered":"2024-03-24T12:10:01.317513+0000","last_clean":"2024-03-24T12:10:01.317513+0000","last_became_active":"2024-03-24T12:09:23.084372+0000","last_became_peered":"2024-03-24T12:09:23.084372+0000","last_unstale":"2024-03-24T12:10:01.317513+0000","last_undegraded":"2024-03-24T12:10:01.317513+0000","last_fullsized":"2024-03-24T12:10:01.317513+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:44:24.541196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317025+0000","last_change":"2024-03-24T12:09:27.149152+0000","last_active":"2024-03-24T12:10:01.317025+0000","last_peered":"2024-03-24T12:10:01.317025+0000","last_clean":"2024-03-24T12:10:01.317025+0000","last_became_active":"2024-03-24T12:09:27.148824+0000","last_became_peered":"2024-03-24T12:09:27.148824+0000","last_unstale":"2024-03-24T12:10:01.317025+0000","last_undegraded":"2024-03-24T12:10:01.317025+0000","last_fullsized":"2024-03-24T12:10:01.317025+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:41:04.422011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.116251+0000","last_change":"2024-03-24T12:09:25.103657+0000","last_active":"2024-03-24T12:10:01.116251+0000","last_peered":"2024-03-24T12:10:01.116251+0000","last_clean":"2024-03-24T12:10:01.116251+0000","last_became_active":"2024-03-24T12:09:25.103201+0000","last_became_peered":"2024-03-24T12:09:25.103201+0000","last_unstale":"2024-03-24T12:10:01.116251+0000","last_undegraded":"2024-03-24T12:10:01.116251+0000","last_fullsized":"2024-03-24T12:10:01.116251+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:21:15.682551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107989+0000","last_change":"2024-03-24T12:09:20.824532+0000","last_active":"2024-03-24T12:10:01.107989+0000","last_peered":"2024-03-24T12:10:01.107989+0000","last_clean":"2024-03-24T12:10:01.107989+0000","last_became_active":"2024-03-24T12:09:20.824171+0000","last_became_peered":"2024-03-24T12:09:20.824171+0000","last_unstale":"2024-03-24T12:10:01.107989+0000","last_undegraded":"2024-03-24T12:10:01.107989+0000","last_fullsized":"2024-03-24T12:10:01.107989+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T23:51:17.104336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"68'11","reported_seq":48,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.319023+0000","last_change":"2024-03-24T12:09:23.143989+0000","last_active":"2024-03-24T12:10:01.319023+0000","last_peered":"2024-03-24T12:10:01.319023+0000","last_clean":"2024-03-24T12:10:01.319023+0000","last_became_active":"2024-03-24T12:09:23.143603+0000","last_became_peered":"2024-03-24T12:09:23.143603+0000","last_unstale":"2024-03-24T12:10:01.319023+0000","last_undegraded":"2024-03-24T12:10:01.319023+0000","last_fullsized":"2024-03-24T12:10:01.319023+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:49:59.464908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.993863+0000","last_change":"2024-03-24T12:09:27.120530+0000","last_active":"2024-03-24T12:10:00.993863+0000","last_peered":"2024-03-24T12:10:00.993863+0000","last_clean":"2024-03-24T12:10:00.993863+0000","last_became_active":"2024-03-24T12:09:27.120213+0000","last_became_peered":"2024-03-24T12:09:27.120213+0000","last_unstale":"2024-03-24T12:10:00.993863+0000","last_undegraded":"2024-03-24T12:10:00.993863+0000","last_fullsized":"2024-03-24T12:10:00.993863+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T23:24:49.314839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317350+0000","last_change":"2024-03-24T12:09:25.103830+0000","last_active":"2024-03-24T12:10:01.317350+0000","last_peered":"2024-03-24T12:10:01.317350+0000","last_clean":"2024-03-24T12:10:01.317350+0000","last_became_active":"2024-03-24T12:09:25.103516+0000","last_became_peered":"2024-03-24T12:09:25.103516+0000","last_unstale":"2024-03-24T12:10:01.317350+0000","last_undegraded":"2024-03-24T12:10:01.317350+0000","last_fullsized":"2024-03-24T12:10:01.317350+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T22:07:14.468870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.116405+0000","last_change":"2024-03-24T12:09:20.825215+0000","last_active":"2024-03-24T12:10:01.116405+0000","last_peered":"2024-03-24T12:10:01.116405+0000","last_clean":"2024-03-24T12:10:01.116405+0000","last_became_active":"2024-03-24T12:09:20.824899+0000","last_became_peered":"2024-03-24T12:09:20.824899+0000","last_unstale":"2024-03-24T12:10:01.116405+0000","last_undegraded":"2024-03-24T12:10:01.116405+0000","last_fullsized":"2024-03-24T12:10:01.116405+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:51:12.685487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317710+0000","last_change":"2024-03-24T12:09:23.086663+0000","last_active":"2024-03-24T12:10:01.317710+0000","last_peered":"2024-03-24T12:10:01.317710+0000","last_clean":"2024-03-24T12:10:01.317710+0000","last_became_active":"2024-03-24T12:09:23.086344+0000","last_became_peered":"2024-03-24T12:09:23.086344+0000","last_unstale":"2024-03-24T12:10:01.317710+0000","last_undegraded":"2024-03-24T12:10:01.317710+0000","last_fullsized":"2024-03-24T12:10:01.317710+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:39:45.014884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.318770+0000","last_change":"2024-03-24T12:09:27.117111+0000","last_active":"2024-03-24T12:10:01.318770+0000","last_peered":"2024-03-24T12:10:01.318770+0000","last_clean":"2024-03-24T12:10:01.318770+0000","last_became_active":"2024-03-24T12:09:27.116792+0000","last_became_peered":"2024-03-24T12:09:27.116792+0000","last_unstale":"2024-03-24T12:10:01.318770+0000","last_undegraded":"2024-03-24T12:10:01.318770+0000","last_fullsized":"2024-03-24T12:10:01.318770+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:34:42.154278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317619+0000","last_change":"2024-03-24T12:09:25.100853+0000","last_active":"2024-03-24T12:10:01.317619+0000","last_peered":"2024-03-24T12:10:01.317619+0000","last_clean":"2024-03-24T12:10:01.317619+0000","last_became_active":"2024-03-24T12:09:25.098585+0000","last_became_peered":"2024-03-24T12:09:25.098585+0000","last_unstale":"2024-03-24T12:10:01.317619+0000","last_undegraded":"2024-03-24T12:10:01.317619+0000","last_fullsized":"2024-03-24T12:10:01.317619+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:53:07.278770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.998323+0000","last_change":"2024-03-24T12:09:20.826612+0000","last_active":"2024-03-24T12:10:00.998323+0000","last_peered":"2024-03-24T12:10:00.998323+0000","last_clean":"2024-03-24T12:10:00.998323+0000","last_became_active":"2024-03-24T12:09:20.826270+0000","last_became_peered":"2024-03-24T12:09:20.826270+0000","last_unstale":"2024-03-24T12:10:00.998323+0000","last_undegraded":"2024-03-24T12:10:00.998323+0000","last_fullsized":"2024-03-24T12:10:00.998323+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:22:08.311526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"68'11","reported_seq":48,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997926+0000","last_change":"2024-03-24T12:09:23.078680+0000","last_active":"2024-03-24T12:10:00.997926+0000","last_peered":"2024-03-24T12:10:00.997926+0000","last_clean":"2024-03-24T12:10:00.997926+0000","last_became_active":"2024-03-24T12:09:23.077788+0000","last_became_peered":"2024-03-24T12:09:23.077788+0000","last_unstale":"2024-03-24T12:10:00.997926+0000","last_undegraded":"2024-03-24T12:10:00.997926+0000","last_fullsized":"2024-03-24T12:10:00.997926+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:49:48.290082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.994580+0000","last_change":"2024-03-24T12:09:27.115226+0000","last_active":"2024-03-24T12:10:00.994580+0000","last_peered":"2024-03-24T12:10:00.994580+0000","last_clean":"2024-03-24T12:10:00.994580+0000","last_became_active":"2024-03-24T12:09:27.114430+0000","last_became_peered":"2024-03-24T12:09:27.114430+0000","last_unstale":"2024-03-24T12:10:00.994580+0000","last_undegraded":"2024-03-24T12:10:00.994580+0000","last_fullsized":"2024-03-24T12:10:00.994580+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:22:41.665035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"68'8","reported_seq":49,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:31.500689+0000","last_change":"2024-03-24T12:09:25.105566+0000","last_active":"2024-03-24T12:10:31.500689+0000","last_peered":"2024-03-24T12:10:31.500689+0000","last_clean":"2024-03-24T12:10:31.500689+0000","last_became_active":"2024-03-24T12:09:25.105262+0000","last_became_peered":"2024-03-24T12:09:25.105262+0000","last_unstale":"2024-03-24T12:10:31.500689+0000","last_undegraded":"2024-03-24T12:10:31.500689+0000","last_fullsized":"2024-03-24T12:10:31.500689+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:17:06.225373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.116023+0000","last_change":"2024-03-24T12:09:21.444535+0000","last_active":"2024-03-24T12:10:01.116023+0000","last_peered":"2024-03-24T12:10:01.116023+0000","last_clean":"2024-03-24T12:10:01.116023+0000","last_became_active":"2024-03-24T12:09:21.444083+0000","last_became_peered":"2024-03-24T12:09:21.444083+0000","last_unstale":"2024-03-24T12:10:01.116023+0000","last_undegraded":"2024-03-24T12:10:01.116023+0000","last_fullsized":"2024-03-24T12:10:01.116023+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T20:35:06.389620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"68'10","reported_seq":44,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317980+0000","last_change":"2024-03-24T12:09:23.144171+0000","last_active":"2024-03-24T12:10:01.317980+0000","last_peered":"2024-03-24T12:10:01.317980+0000","last_clean":"2024-03-24T12:10:01.317980+0000","last_became_active":"2024-03-24T12:09:23.143744+0000","last_became_peered":"2024-03-24T12:09:23.143744+0000","last_unstale":"2024-03-24T12:10:01.317980+0000","last_undegraded":"2024-03-24T12:10:01.317980+0000","last_fullsized":"2024-03-24T12:10:01.317980+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:31:27.373390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.998801+0000","last_change":"2024-03-24T12:09:27.112487+0000","last_active":"2024-03-24T12:10:00.998801+0000","last_peered":"2024-03-24T12:10:00.998801+0000","last_clean":"2024-03-24T12:10:00.998801+0000","last_became_active":"2024-03-24T12:09:27.112179+0000","last_became_peered":"2024-03-24T12:09:27.112179+0000","last_unstale":"2024-03-24T12:10:00.998801+0000","last_undegraded":"2024-03-24T12:10:00.998801+0000","last_fullsized":"2024-03-24T12:10:00.998801+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T20:25:10.244533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"68'8","reported_seq":52,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:31.500622+0000","last_change":"2024-03-24T12:09:25.100165+0000","last_active":"2024-03-24T12:10:31.500622+0000","last_peered":"2024-03-24T12:10:31.500622+0000","last_clean":"2024-03-24T12:10:31.500622+0000","last_became_active":"2024-03-24T12:09:25.099826+0000","last_became_peered":"2024-03-24T12:09:25.099826+0000","last_unstale":"2024-03-24T12:10:31.500622+0000","last_undegraded":"2024-03-24T12:10:31.500622+0000","last_fullsized":"2024-03-24T12:10:31.500622+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T22:45:53.795860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.994195+0000","last_change":"2024-03-24T12:09:20.827524+0000","last_active":"2024-03-24T12:10:00.994195+0000","last_peered":"2024-03-24T12:10:00.994195+0000","last_clean":"2024-03-24T12:10:00.994195+0000","last_became_active":"2024-03-24T12:09:20.827209+0000","last_became_peered":"2024-03-24T12:09:20.827209+0000","last_unstale":"2024-03-24T12:10:00.994195+0000","last_undegraded":"2024-03-24T12:10:00.994195+0000","last_fullsized":"2024-03-24T12:10:00.994195+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:20:53.400737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.108014+0000","last_change":"2024-03-24T12:09:23.147034+0000","last_active":"2024-03-24T12:10:01.108014+0000","last_peered":"2024-03-24T12:10:01.108014+0000","last_clean":"2024-03-24T12:10:01.108014+0000","last_became_active":"2024-03-24T12:09:23.146646+0000","last_became_peered":"2024-03-24T12:09:23.146646+0000","last_unstale":"2024-03-24T12:10:01.108014+0000","last_undegraded":"2024-03-24T12:10:01.108014+0000","last_fullsized":"2024-03-24T12:10:01.108014+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:15:31.126526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.113629+0000","last_change":"2024-03-24T12:09:27.112023+0000","last_active":"2024-03-24T12:10:01.113629+0000","last_peered":"2024-03-24T12:10:01.113629+0000","last_clean":"2024-03-24T12:10:01.113629+0000","last_became_active":"2024-03-24T12:09:27.111576+0000","last_became_peered":"2024-03-24T12:09:27.111576+0000","last_unstale":"2024-03-24T12:10:01.113629+0000","last_undegraded":"2024-03-24T12:10:01.113629+0000","last_fullsized":"2024-03-24T12:10:01.113629+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:54:44.964282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.318248+0000","last_change":"2024-03-24T12:09:25.106357+0000","last_active":"2024-03-24T12:10:01.318248+0000","last_peered":"2024-03-24T12:10:01.318248+0000","last_clean":"2024-03-24T12:10:01.318248+0000","last_became_active":"2024-03-24T12:09:25.106002+0000","last_became_peered":"2024-03-24T12:09:25.106002+0000","last_unstale":"2024-03-24T12:10:01.318248+0000","last_undegraded":"2024-03-24T12:10:01.318248+0000","last_fullsized":"2024-03-24T12:10:01.318248+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:12:42.285452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.115958+0000","last_change":"2024-03-24T12:09:20.826435+0000","last_active":"2024-03-24T12:10:01.115958+0000","last_peered":"2024-03-24T12:10:01.115958+0000","last_clean":"2024-03-24T12:10:01.115958+0000","last_became_active":"2024-03-24T12:09:20.826156+0000","last_became_peered":"2024-03-24T12:09:20.826156+0000","last_unstale":"2024-03-24T12:10:01.115958+0000","last_undegraded":"2024-03-24T12:10:01.115958+0000","last_fullsized":"2024-03-24T12:10:01.115958+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T22:45:03.024321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.995488+0000","last_change":"2024-03-24T12:09:23.144571+0000","last_active":"2024-03-24T12:10:00.995488+0000","last_peered":"2024-03-24T12:10:00.995488+0000","last_clean":"2024-03-24T12:10:00.995488+0000","last_became_active":"2024-03-24T12:09:23.144231+0000","last_became_peered":"2024-03-24T12:09:23.144231+0000","last_unstale":"2024-03-24T12:10:00.995488+0000","last_undegraded":"2024-03-24T12:10:00.995488+0000","last_fullsized":"2024-03-24T12:10:00.995488+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:33:42.836952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.116342+0000","last_change":"2024-03-24T12:09:27.149704+0000","last_active":"2024-03-24T12:10:01.116342+0000","last_peered":"2024-03-24T12:10:01.116342+0000","last_clean":"2024-03-24T12:10:01.116342+0000","last_became_active":"2024-03-24T12:09:27.149378+0000","last_became_peered":"2024-03-24T12:09:27.149378+0000","last_unstale":"2024-03-24T12:10:01.116342+0000","last_undegraded":"2024-03-24T12:10:01.116342+0000","last_fullsized":"2024-03-24T12:10:01.116342+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:42:40.151707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107981+0000","last_change":"2024-03-24T12:09:25.102940+0000","last_active":"2024-03-24T12:10:01.107981+0000","last_peered":"2024-03-24T12:10:01.107981+0000","last_clean":"2024-03-24T12:10:01.107981+0000","last_became_active":"2024-03-24T12:09:25.102625+0000","last_became_peered":"2024-03-24T12:09:25.102625+0000","last_unstale":"2024-03-24T12:10:01.107981+0000","last_undegraded":"2024-03-24T12:10:01.107981+0000","last_fullsized":"2024-03-24T12:10:01.107981+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:00:23.241037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.316645+0000","last_change":"2024-03-24T12:09:20.821247+0000","last_active":"2024-03-24T12:10:01.316645+0000","last_peered":"2024-03-24T12:10:01.316645+0000","last_clean":"2024-03-24T12:10:01.316645+0000","last_became_active":"2024-03-24T12:09:20.820941+0000","last_became_peered":"2024-03-24T12:09:20.820941+0000","last_unstale":"2024-03-24T12:10:01.316645+0000","last_undegraded":"2024-03-24T12:10:01.316645+0000","last_fullsized":"2024-03-24T12:10:01.316645+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:34:38.848780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"68'6","reported_seq":38,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.318690+0000","last_change":"2024-03-24T12:09:23.145277+0000","last_active":"2024-03-24T12:10:01.318690+0000","last_peered":"2024-03-24T12:10:01.318690+0000","last_clean":"2024-03-24T12:10:01.318690+0000","last_became_active":"2024-03-24T12:09:23.144971+0000","last_became_peered":"2024-03-24T12:09:23.144971+0000","last_unstale":"2024-03-24T12:10:01.318690+0000","last_undegraded":"2024-03-24T12:10:01.318690+0000","last_fullsized":"2024-03-24T12:10:01.318690+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T15:54:44.221081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107759+0000","last_change":"2024-03-24T12:09:27.150904+0000","last_active":"2024-03-24T12:10:01.107759+0000","last_peered":"2024-03-24T12:10:01.107759+0000","last_clean":"2024-03-24T12:10:01.107759+0000","last_became_active":"2024-03-24T12:09:27.150531+0000","last_became_peered":"2024-03-24T12:09:27.150531+0000","last_unstale":"2024-03-24T12:10:01.107759+0000","last_undegraded":"2024-03-24T12:10:01.107759+0000","last_fullsized":"2024-03-24T12:10:01.107759+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:36:29.239350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"61'2","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.116682+0000","last_change":"2024-03-24T12:09:20.827574+0000","last_active":"2024-03-24T12:10:01.116682+0000","last_peered":"2024-03-24T12:10:01.116682+0000","last_clean":"2024-03-24T12:10:01.116682+0000","last_became_active":"2024-03-24T12:09:20.827179+0000","last_became_peered":"2024-03-24T12:09:20.827179+0000","last_unstale":"2024-03-24T12:10:01.116682+0000","last_undegraded":"2024-03-24T12:10:01.116682+0000","last_fullsized":"2024-03-24T12:10:01.116682+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:11:47.157188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"68'15","reported_seq":54,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107293+0000","last_change":"2024-03-24T12:09:23.145502+0000","last_active":"2024-03-24T12:10:01.107293+0000","last_peered":"2024-03-24T12:10:01.107293+0000","last_clean":"2024-03-24T12:10:01.107293+0000","last_became_active":"2024-03-24T12:09:23.145146+0000","last_became_peered":"2024-03-24T12:09:23.145146+0000","last_unstale":"2024-03-24T12:10:01.107293+0000","last_undegraded":"2024-03-24T12:10:01.107293+0000","last_fullsized":"2024-03-24T12:10:01.107293+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T20:29:27.507482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"68'8","reported_seq":49,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:31.500713+0000","last_change":"2024-03-24T12:09:25.103540+0000","last_active":"2024-03-24T12:10:31.500713+0000","last_peered":"2024-03-24T12:10:31.500713+0000","last_clean":"2024-03-24T12:10:31.500713+0000","last_became_active":"2024-03-24T12:09:25.103180+0000","last_became_peered":"2024-03-24T12:09:25.103180+0000","last_unstale":"2024-03-24T12:10:31.500713+0000","last_undegraded":"2024-03-24T12:10:31.500713+0000","last_fullsized":"2024-03-24T12:10:31.500713+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:04:02.081956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317132+0000","last_change":"2024-03-24T12:09:27.116381+0000","last_active":"2024-03-24T12:10:01.317132+0000","last_peered":"2024-03-24T12:10:01.317132+0000","last_clean":"2024-03-24T12:10:01.317132+0000","last_became_active":"2024-03-24T12:09:27.115964+0000","last_became_peered":"2024-03-24T12:09:27.115964+0000","last_unstale":"2024-03-24T12:10:01.317132+0000","last_undegraded":"2024-03-24T12:10:01.317132+0000","last_fullsized":"2024-03-24T12:10:01.317132+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:16:03.870130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.116590+0000","last_change":"2024-03-24T12:09:20.825958+0000","last_active":"2024-03-24T12:10:01.116590+0000","last_peered":"2024-03-24T12:10:01.116590+0000","last_clean":"2024-03-24T12:10:01.116590+0000","last_became_active":"2024-03-24T12:09:20.825639+0000","last_became_peered":"2024-03-24T12:09:20.825639+0000","last_unstale":"2024-03-24T12:10:01.116590+0000","last_undegraded":"2024-03-24T12:10:01.116590+0000","last_fullsized":"2024-03-24T12:10:01.116590+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T22:09:13.396812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"68'12","reported_seq":70,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997380+0000","last_change":"2024-03-24T12:09:46.968266+0000","last_active":"2024-03-24T12:10:00.997380+0000","last_peered":"2024-03-24T12:10:00.997380+0000","last_clean":"2024-03-24T12:10:00.997380+0000","last_became_active":"2024-03-24T12:09:46.960585+0000","last_became_peered":"2024-03-24T12:09:46.960585+0000","last_unstale":"2024-03-24T12:10:00.997380+0000","last_undegraded":"2024-03-24T12:10:00.997380+0000","last_fullsized":"2024-03-24T12:10:00.997380+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:37:34.452248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.112200+0000","last_change":"2024-03-24T12:09:25.102613+0000","last_active":"2024-03-24T12:10:01.112200+0000","last_peered":"2024-03-24T12:10:01.112200+0000","last_clean":"2024-03-24T12:10:01.112200+0000","last_became_active":"2024-03-24T12:09:25.102224+0000","last_became_peered":"2024-03-24T12:09:25.102224+0000","last_unstale":"2024-03-24T12:10:01.112200+0000","last_undegraded":"2024-03-24T12:10:01.112200+0000","last_fullsized":"2024-03-24T12:10:01.112200+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T23:04:06.272333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.115781+0000","last_change":"2024-03-24T12:09:27.119327+0000","last_active":"2024-03-24T12:10:01.115781+0000","last_peered":"2024-03-24T12:10:01.115781+0000","last_clean":"2024-03-24T12:10:01.115781+0000","last_became_active":"2024-03-24T12:09:27.119062+0000","last_became_peered":"2024-03-24T12:09:27.119062+0000","last_unstale":"2024-03-24T12:10:01.115781+0000","last_undegraded":"2024-03-24T12:10:01.115781+0000","last_fullsized":"2024-03-24T12:10:01.115781+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:54:06.558279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.112699+0000","last_change":"2024-03-24T12:09:25.100273+0000","last_active":"2024-03-24T12:10:01.112699+0000","last_peered":"2024-03-24T12:10:01.112699+0000","last_clean":"2024-03-24T12:10:01.112699+0000","last_became_active":"2024-03-24T12:09:25.099235+0000","last_became_peered":"2024-03-24T12:09:25.099235+0000","last_unstale":"2024-03-24T12:10:01.112699+0000","last_undegraded":"2024-03-24T12:10:01.112699+0000","last_fullsized":"2024-03-24T12:10:01.112699+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:42:13.058472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.116588+0000","last_change":"2024-03-24T12:09:20.829111+0000","last_active":"2024-03-24T12:10:01.116588+0000","last_peered":"2024-03-24T12:10:01.116588+0000","last_clean":"2024-03-24T12:10:01.116588+0000","last_became_active":"2024-03-24T12:09:20.828736+0000","last_became_peered":"2024-03-24T12:09:20.828736+0000","last_unstale":"2024-03-24T12:10:01.116588+0000","last_undegraded":"2024-03-24T12:10:01.116588+0000","last_fullsized":"2024-03-24T12:10:01.116588+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T20:39:57.081827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"68'19","reported_seq":60,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.316828+0000","last_change":"2024-03-24T12:09:23.144173+0000","last_active":"2024-03-24T12:10:01.316828+0000","last_peered":"2024-03-24T12:10:01.316828+0000","last_clean":"2024-03-24T12:10:01.316828+0000","last_became_active":"2024-03-24T12:09:23.143716+0000","last_became_peered":"2024-03-24T12:09:23.143716+0000","last_unstale":"2024-03-24T12:10:01.316828+0000","last_undegraded":"2024-03-24T12:10:01.316828+0000","last_fullsized":"2024-03-24T12:10:01.316828+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:50:54.248390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.993951+0000","last_change":"2024-03-24T12:09:27.112975+0000","last_active":"2024-03-24T12:10:00.993951+0000","last_peered":"2024-03-24T12:10:00.993951+0000","last_clean":"2024-03-24T12:10:00.993951+0000","last_became_active":"2024-03-24T12:09:27.112571+0000","last_became_peered":"2024-03-24T12:09:27.112571+0000","last_unstale":"2024-03-24T12:10:00.993951+0000","last_undegraded":"2024-03-24T12:10:00.993951+0000","last_fullsized":"2024-03-24T12:10:00.993951+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:26:30.958462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.112726+0000","last_change":"2024-03-24T12:09:25.095768+0000","last_active":"2024-03-24T12:10:01.112726+0000","last_peered":"2024-03-24T12:10:01.112726+0000","last_clean":"2024-03-24T12:10:01.112726+0000","last_became_active":"2024-03-24T12:09:25.095444+0000","last_became_peered":"2024-03-24T12:09:25.095444+0000","last_unstale":"2024-03-24T12:10:01.112726+0000","last_undegraded":"2024-03-24T12:10:01.112726+0000","last_fullsized":"2024-03-24T12:10:01.112726+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T20:31:15.096425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107629+0000","last_change":"2024-03-24T12:09:20.819784+0000","last_active":"2024-03-24T12:10:01.107629+0000","last_peered":"2024-03-24T12:10:01.107629+0000","last_clean":"2024-03-24T12:10:01.107629+0000","last_became_active":"2024-03-24T12:09:20.819447+0000","last_became_peered":"2024-03-24T12:09:20.819447+0000","last_unstale":"2024-03-24T12:10:01.107629+0000","last_undegraded":"2024-03-24T12:10:01.107629+0000","last_fullsized":"2024-03-24T12:10:01.107629+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:13:09.930323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.995793+0000","last_change":"2024-03-24T12:09:23.080500+0000","last_active":"2024-03-24T12:10:00.995793+0000","last_peered":"2024-03-24T12:10:00.995793+0000","last_clean":"2024-03-24T12:10:00.995793+0000","last_became_active":"2024-03-24T12:09:23.079806+0000","last_became_peered":"2024-03-24T12:09:23.079806+0000","last_unstale":"2024-03-24T12:10:00.995793+0000","last_undegraded":"2024-03-24T12:10:00.995793+0000","last_fullsized":"2024-03-24T12:10:00.995793+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:27:35.278225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317861+0000","last_change":"2024-03-24T12:09:27.114005+0000","last_active":"2024-03-24T12:10:01.317861+0000","last_peered":"2024-03-24T12:10:01.317861+0000","last_clean":"2024-03-24T12:10:01.317861+0000","last_became_active":"2024-03-24T12:09:27.113656+0000","last_became_peered":"2024-03-24T12:09:27.113656+0000","last_unstale":"2024-03-24T12:10:01.317861+0000","last_undegraded":"2024-03-24T12:10:01.317861+0000","last_fullsized":"2024-03-24T12:10:01.317861+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:30:35.688034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"64'2","reported_seq":44,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107484+0000","last_change":"2024-03-24T12:09:24.090689+0000","last_active":"2024-03-24T12:10:01.107484+0000","last_peered":"2024-03-24T12:10:01.107484+0000","last_clean":"2024-03-24T12:10:01.107484+0000","last_became_active":"2024-03-24T12:09:19.818583+0000","last_became_peered":"2024-03-24T12:09:19.818583+0000","last_unstale":"2024-03-24T12:10:01.107484+0000","last_undegraded":"2024-03-24T12:10:01.107484+0000","last_fullsized":"2024-03-24T12:10:01.107484+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:18.800675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:18.800675+0000","last_clean_scrub_stamp":"2024-03-24T12:09:18.800675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:44:54.337302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022609079999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"61'1","reported_seq":39,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997659+0000","last_change":"2024-03-24T12:09:20.820979+0000","last_active":"2024-03-24T12:10:00.997659+0000","last_peered":"2024-03-24T12:10:00.997659+0000","last_clean":"2024-03-24T12:10:00.997659+0000","last_became_active":"2024-03-24T12:09:20.820649+0000","last_became_peered":"2024-03-24T12:09:20.820649+0000","last_unstale":"2024-03-24T12:10:00.997659+0000","last_undegraded":"2024-03-24T12:10:00.997659+0000","last_fullsized":"2024-03-24T12:10:00.997659+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:16:02.180081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":875,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"68'28","reported_seq":93,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:31.500402+0000","last_change":"2024-03-24T12:09:23.088242+0000","last_active":"2024-03-24T12:10:31.500402+0000","last_peered":"2024-03-24T12:10:31.500402+0000","last_clean":"2024-03-24T12:10:31.500402+0000","last_became_active":"2024-03-24T12:09:23.087959+0000","last_became_peered":"2024-03-24T12:09:23.087959+0000","last_unstale":"2024-03-24T12:10:31.500402+0000","last_undegraded":"2024-03-24T12:10:31.500402+0000","last_fullsized":"2024-03-24T12:10:31.500402+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:26:06.935634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.994408+0000","last_change":"2024-03-24T12:09:25.105808+0000","last_active":"2024-03-24T12:10:00.994408+0000","last_peered":"2024-03-24T12:10:00.994408+0000","last_clean":"2024-03-24T12:10:00.994408+0000","last_became_active":"2024-03-24T12:09:25.105394+0000","last_became_peered":"2024-03-24T12:09:25.105394+0000","last_unstale":"2024-03-24T12:10:00.994408+0000","last_undegraded":"2024-03-24T12:10:00.994408+0000","last_fullsized":"2024-03-24T12:10:00.994408+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T23:39:21.837785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.116678+0000","last_change":"2024-03-24T12:09:27.151051+0000","last_active":"2024-03-24T12:10:01.116678+0000","last_peered":"2024-03-24T12:10:01.116678+0000","last_clean":"2024-03-24T12:10:01.116678+0000","last_became_active":"2024-03-24T12:09:27.150216+0000","last_became_peered":"2024-03-24T12:09:27.150216+0000","last_unstale":"2024-03-24T12:10:01.116678+0000","last_undegraded":"2024-03-24T12:10:01.116678+0000","last_fullsized":"2024-03-24T12:10:01.116678+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:24:33.029899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.316866+0000","last_change":"2024-03-24T12:09:21.445065+0000","last_active":"2024-03-24T12:10:01.316866+0000","last_peered":"2024-03-24T12:10:01.316866+0000","last_clean":"2024-03-24T12:10:01.316866+0000","last_became_active":"2024-03-24T12:09:21.444217+0000","last_became_peered":"2024-03-24T12:09:21.444217+0000","last_unstale":"2024-03-24T12:10:01.316866+0000","last_undegraded":"2024-03-24T12:10:01.316866+0000","last_fullsized":"2024-03-24T12:10:01.316866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:21:10.451340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.112433+0000","last_change":"2024-03-24T12:09:24.084016+0000","last_active":"2024-03-24T12:10:01.112433+0000","last_peered":"2024-03-24T12:10:01.112433+0000","last_clean":"2024-03-24T12:10:01.112433+0000","last_became_active":"2024-03-24T12:09:19.816551+0000","last_became_peered":"2024-03-24T12:09:19.816551+0000","last_unstale":"2024-03-24T12:10:01.112433+0000","last_undegraded":"2024-03-24T12:10:01.112433+0000","last_fullsized":"2024-03-24T12:10:01.112433+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:18.800675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:18.800675+0000","last_clean_scrub_stamp":"2024-03-24T12:09:18.800675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:00:19.277567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065894900000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"68'13","reported_seq":56,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.318235+0000","last_change":"2024-03-24T12:09:23.081979+0000","last_active":"2024-03-24T12:10:01.318235+0000","last_peered":"2024-03-24T12:10:01.318235+0000","last_clean":"2024-03-24T12:10:01.318235+0000","last_became_active":"2024-03-24T12:09:23.080958+0000","last_became_peered":"2024-03-24T12:09:23.080958+0000","last_unstale":"2024-03-24T12:10:01.318235+0000","last_undegraded":"2024-03-24T12:10:01.318235+0000","last_fullsized":"2024-03-24T12:10:01.318235+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:07:11.319443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.112790+0000","last_change":"2024-03-24T12:09:25.100039+0000","last_active":"2024-03-24T12:10:01.112790+0000","last_peered":"2024-03-24T12:10:01.112790+0000","last_clean":"2024-03-24T12:10:01.112790+0000","last_became_active":"2024-03-24T12:09:25.098643+0000","last_became_peered":"2024-03-24T12:09:25.098643+0000","last_unstale":"2024-03-24T12:10:01.112790+0000","last_undegraded":"2024-03-24T12:10:01.112790+0000","last_fullsized":"2024-03-24T12:10:01.112790+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:40:03.778416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107820+0000","last_change":"2024-03-24T12:09:27.118123+0000","last_active":"2024-03-24T12:10:01.107820+0000","last_peered":"2024-03-24T12:10:01.107820+0000","last_clean":"2024-03-24T12:10:01.107820+0000","last_became_active":"2024-03-24T12:09:27.117788+0000","last_became_peered":"2024-03-24T12:09:27.117788+0000","last_unstale":"2024-03-24T12:10:01.107820+0000","last_undegraded":"2024-03-24T12:10:01.107820+0000","last_fullsized":"2024-03-24T12:10:01.107820+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:23:24.436351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":278,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:03.098710+0000","last_change":"2024-03-24T12:09:10.475623+0000","last_active":"2024-03-24T12:10:03.098710+0000","last_peered":"2024-03-24T12:10:03.098710+0000","last_clean":"2024-03-24T12:10:03.098710+0000","last_became_active":"2024-03-24T12:09:10.475313+0000","last_became_peered":"2024-03-24T12:09:10.475313+0000","last_unstale":"2024-03-24T12:10:03.098710+0000","last_undegraded":"2024-03-24T12:10:03.098710+0000","last_fullsized":"2024-03-24T12:10:03.098710+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:07:02.602210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:07:02.602210+0000","last_clean_scrub_stamp":"2024-03-24T12:07:02.602210+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:53:43.283275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317252+0000","last_change":"2024-03-24T12:09:20.821957+0000","last_active":"2024-03-24T12:10:01.317252+0000","last_peered":"2024-03-24T12:10:01.317252+0000","last_clean":"2024-03-24T12:10:01.317252+0000","last_became_active":"2024-03-24T12:09:20.821651+0000","last_became_peered":"2024-03-24T12:09:20.821651+0000","last_unstale":"2024-03-24T12:10:01.317252+0000","last_undegraded":"2024-03-24T12:10:01.317252+0000","last_fullsized":"2024-03-24T12:10:01.317252+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:20:03.531446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"68'16","reported_seq":67,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:31.500703+0000","last_change":"2024-03-24T12:09:23.145396+0000","last_active":"2024-03-24T12:10:31.500703+0000","last_peered":"2024-03-24T12:10:31.500703+0000","last_clean":"2024-03-24T12:10:31.500703+0000","last_became_active":"2024-03-24T12:09:23.144993+0000","last_became_peered":"2024-03-24T12:09:23.144993+0000","last_unstale":"2024-03-24T12:10:31.500703+0000","last_undegraded":"2024-03-24T12:10:31.500703+0000","last_fullsized":"2024-03-24T12:10:31.500703+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:22:14.043318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.116407+0000","last_change":"2024-03-24T12:09:25.104997+0000","last_active":"2024-03-24T12:10:01.116407+0000","last_peered":"2024-03-24T12:10:01.116407+0000","last_clean":"2024-03-24T12:10:01.116407+0000","last_became_active":"2024-03-24T12:09:25.104683+0000","last_became_peered":"2024-03-24T12:09:25.104683+0000","last_unstale":"2024-03-24T12:10:01.116407+0000","last_undegraded":"2024-03-24T12:10:01.116407+0000","last_fullsized":"2024-03-24T12:10:01.116407+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:31:23.470732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317877+0000","last_change":"2024-03-24T12:09:27.117597+0000","last_active":"2024-03-24T12:10:01.317877+0000","last_peered":"2024-03-24T12:10:01.317877+0000","last_clean":"2024-03-24T12:10:01.317877+0000","last_became_active":"2024-03-24T12:09:27.117243+0000","last_became_peered":"2024-03-24T12:09:27.117243+0000","last_unstale":"2024-03-24T12:10:01.317877+0000","last_undegraded":"2024-03-24T12:10:01.317877+0000","last_fullsized":"2024-03-24T12:10:01.317877+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:18:16.692691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.994134+0000","last_change":"2024-03-24T12:09:20.822275+0000","last_active":"2024-03-24T12:10:00.994134+0000","last_peered":"2024-03-24T12:10:00.994134+0000","last_clean":"2024-03-24T12:10:00.994134+0000","last_became_active":"2024-03-24T12:09:20.821920+0000","last_became_peered":"2024-03-24T12:09:20.821920+0000","last_unstale":"2024-03-24T12:10:00.994134+0000","last_undegraded":"2024-03-24T12:10:00.994134+0000","last_fullsized":"2024-03-24T12:10:00.994134+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:52:20.544068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"69'5","reported_seq":114,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:35.936400+0000","last_change":"2024-03-24T12:09:24.089836+0000","last_active":"2024-03-24T12:10:35.936400+0000","last_peered":"2024-03-24T12:10:35.936400+0000","last_clean":"2024-03-24T12:10:35.936400+0000","last_became_active":"2024-03-24T12:09:19.818613+0000","last_became_peered":"2024-03-24T12:09:19.818613+0000","last_unstale":"2024-03-24T12:10:35.936400+0000","last_undegraded":"2024-03-24T12:10:35.936400+0000","last_fullsized":"2024-03-24T12:10:35.936400+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:18.800675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:18.800675+0000","last_clean_scrub_stamp":"2024-03-24T12:09:18.800675+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:32:33.066054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042976100000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":69,"num_read_kb":64,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"68'12","reported_seq":47,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.994676+0000","last_change":"2024-03-24T12:09:23.079972+0000","last_active":"2024-03-24T12:10:00.994676+0000","last_peered":"2024-03-24T12:10:00.994676+0000","last_clean":"2024-03-24T12:10:00.994676+0000","last_became_active":"2024-03-24T12:09:23.079579+0000","last_became_peered":"2024-03-24T12:09:23.079579+0000","last_unstale":"2024-03-24T12:10:00.994676+0000","last_undegraded":"2024-03-24T12:10:00.994676+0000","last_fullsized":"2024-03-24T12:10:00.994676+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-26T00:01:44.813315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.108158+0000","last_change":"2024-03-24T12:09:25.106326+0000","last_active":"2024-03-24T12:10:01.108158+0000","last_peered":"2024-03-24T12:10:01.108158+0000","last_clean":"2024-03-24T12:10:01.108158+0000","last_became_active":"2024-03-24T12:09:25.106033+0000","last_became_peered":"2024-03-24T12:09:25.106033+0000","last_unstale":"2024-03-24T12:10:01.108158+0000","last_undegraded":"2024-03-24T12:10:01.108158+0000","last_fullsized":"2024-03-24T12:10:01.108158+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:10:38.677188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.316932+0000","last_change":"2024-03-24T12:09:27.149981+0000","last_active":"2024-03-24T12:10:01.316932+0000","last_peered":"2024-03-24T12:10:01.316932+0000","last_clean":"2024-03-24T12:10:01.316932+0000","last_became_active":"2024-03-24T12:09:27.149577+0000","last_became_peered":"2024-03-24T12:09:27.149577+0000","last_unstale":"2024-03-24T12:10:01.316932+0000","last_undegraded":"2024-03-24T12:10:01.316932+0000","last_fullsized":"2024-03-24T12:10:01.316932+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T22:29:28.129633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.318210+0000","last_change":"2024-03-24T12:09:21.444784+0000","last_active":"2024-03-24T12:10:01.318210+0000","last_peered":"2024-03-24T12:10:01.318210+0000","last_clean":"2024-03-24T12:10:01.318210+0000","last_became_active":"2024-03-24T12:09:21.444227+0000","last_became_peered":"2024-03-24T12:09:21.444227+0000","last_unstale":"2024-03-24T12:10:01.318210+0000","last_undegraded":"2024-03-24T12:10:01.318210+0000","last_fullsized":"2024-03-24T12:10:01.318210+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:17:17.206900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"68'19","reported_seq":65,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.995386+0000","last_change":"2024-03-24T12:09:23.144276+0000","last_active":"2024-03-24T12:10:00.995386+0000","last_peered":"2024-03-24T12:10:00.995386+0000","last_clean":"2024-03-24T12:10:00.995386+0000","last_became_active":"2024-03-24T12:09:23.143963+0000","last_became_peered":"2024-03-24T12:09:23.143963+0000","last_unstale":"2024-03-24T12:10:00.995386+0000","last_undegraded":"2024-03-24T12:10:00.995386+0000","last_fullsized":"2024-03-24T12:10:00.995386+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:17:08.898988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.316852+0000","last_change":"2024-03-24T12:09:25.097995+0000","last_active":"2024-03-24T12:10:01.316852+0000","last_peered":"2024-03-24T12:10:01.316852+0000","last_clean":"2024-03-24T12:10:01.316852+0000","last_became_active":"2024-03-24T12:09:25.097693+0000","last_became_peered":"2024-03-24T12:09:25.097693+0000","last_unstale":"2024-03-24T12:10:01.316852+0000","last_undegraded":"2024-03-24T12:10:01.316852+0000","last_fullsized":"2024-03-24T12:10:01.316852+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:31:42.256234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.993874+0000","last_change":"2024-03-24T12:09:27.119367+0000","last_active":"2024-03-24T12:10:00.993874+0000","last_peered":"2024-03-24T12:10:00.993874+0000","last_clean":"2024-03-24T12:10:00.993874+0000","last_became_active":"2024-03-24T12:09:27.119064+0000","last_became_peered":"2024-03-24T12:09:27.119064+0000","last_unstale":"2024-03-24T12:10:00.993874+0000","last_undegraded":"2024-03-24T12:10:00.993874+0000","last_fullsized":"2024-03-24T12:10:00.993874+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:41:48.578066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107853+0000","last_change":"2024-03-24T12:09:21.444520+0000","last_active":"2024-03-24T12:10:01.107853+0000","last_peered":"2024-03-24T12:10:01.107853+0000","last_clean":"2024-03-24T12:10:01.107853+0000","last_became_active":"2024-03-24T12:09:21.444158+0000","last_became_peered":"2024-03-24T12:09:21.444158+0000","last_unstale":"2024-03-24T12:10:01.107853+0000","last_undegraded":"2024-03-24T12:10:01.107853+0000","last_fullsized":"2024-03-24T12:10:01.107853+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T15:57:35.914727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"68'10","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.112399+0000","last_change":"2024-03-24T12:09:46.969753+0000","last_active":"2024-03-24T12:10:01.112399+0000","last_peered":"2024-03-24T12:10:01.112399+0000","last_clean":"2024-03-24T12:10:01.112399+0000","last_became_active":"2024-03-24T12:09:46.961937+0000","last_became_peered":"2024-03-24T12:09:46.961937+0000","last_unstale":"2024-03-24T12:10:01.112399+0000","last_undegraded":"2024-03-24T12:10:01.112399+0000","last_fullsized":"2024-03-24T12:10:01.112399+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:16:34.746853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"68'8","reported_seq":49,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:31.500438+0000","last_change":"2024-03-24T12:09:25.102930+0000","last_active":"2024-03-24T12:10:31.500438+0000","last_peered":"2024-03-24T12:10:31.500438+0000","last_clean":"2024-03-24T12:10:31.500438+0000","last_became_active":"2024-03-24T12:09:25.102575+0000","last_became_peered":"2024-03-24T12:09:25.102575+0000","last_unstale":"2024-03-24T12:10:31.500438+0000","last_undegraded":"2024-03-24T12:10:31.500438+0000","last_fullsized":"2024-03-24T12:10:31.500438+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:36:26.190463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.116271+0000","last_change":"2024-03-24T12:09:27.150769+0000","last_active":"2024-03-24T12:10:01.116271+0000","last_peered":"2024-03-24T12:10:01.116271+0000","last_clean":"2024-03-24T12:10:01.116271+0000","last_became_active":"2024-03-24T12:09:27.150362+0000","last_became_peered":"2024-03-24T12:09:27.150362+0000","last_unstale":"2024-03-24T12:10:01.116271+0000","last_undegraded":"2024-03-24T12:10:01.116271+0000","last_fullsized":"2024-03-24T12:10:01.116271+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:57:32.489010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"61'1","reported_seq":34,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.994810+0000","last_change":"2024-03-24T12:09:20.820873+0000","last_active":"2024-03-24T12:10:00.994810+0000","last_peered":"2024-03-24T12:10:00.994810+0000","last_clean":"2024-03-24T12:10:00.994810+0000","last_became_active":"2024-03-24T12:09:20.819577+0000","last_became_peered":"2024-03-24T12:09:20.819577+0000","last_unstale":"2024-03-24T12:10:00.994810+0000","last_undegraded":"2024-03-24T12:10:00.994810+0000","last_fullsized":"2024-03-24T12:10:00.994810+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:42:49.044131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"68'14","reported_seq":50,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997256+0000","last_change":"2024-03-24T12:09:23.077103+0000","last_active":"2024-03-24T12:10:00.997256+0000","last_peered":"2024-03-24T12:10:00.997256+0000","last_clean":"2024-03-24T12:10:00.997256+0000","last_became_active":"2024-03-24T12:09:23.076572+0000","last_became_peered":"2024-03-24T12:09:23.076572+0000","last_unstale":"2024-03-24T12:10:00.997256+0000","last_undegraded":"2024-03-24T12:10:00.997256+0000","last_fullsized":"2024-03-24T12:10:00.997256+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T15:08:15.084609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"68'8","reported_seq":49,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:31.500393+0000","last_change":"2024-03-24T12:09:25.102449+0000","last_active":"2024-03-24T12:10:31.500393+0000","last_peered":"2024-03-24T12:10:31.500393+0000","last_clean":"2024-03-24T12:10:31.500393+0000","last_became_active":"2024-03-24T12:09:25.102099+0000","last_became_peered":"2024-03-24T12:09:25.102099+0000","last_unstale":"2024-03-24T12:10:31.500393+0000","last_undegraded":"2024-03-24T12:10:31.500393+0000","last_fullsized":"2024-03-24T12:10:31.500393+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T15:21:24.119081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997142+0000","last_change":"2024-03-24T12:09:27.116459+0000","last_active":"2024-03-24T12:10:00.997142+0000","last_peered":"2024-03-24T12:10:00.997142+0000","last_clean":"2024-03-24T12:10:00.997142+0000","last_became_active":"2024-03-24T12:09:27.116175+0000","last_became_peered":"2024-03-24T12:09:27.116175+0000","last_unstale":"2024-03-24T12:10:00.997142+0000","last_undegraded":"2024-03-24T12:10:00.997142+0000","last_fullsized":"2024-03-24T12:10:00.997142+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T23:10:56.728970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.318141+0000","last_change":"2024-03-24T12:09:20.828927+0000","last_active":"2024-03-24T12:10:01.318141+0000","last_peered":"2024-03-24T12:10:01.318141+0000","last_clean":"2024-03-24T12:10:01.318141+0000","last_became_active":"2024-03-24T12:09:20.828616+0000","last_became_peered":"2024-03-24T12:09:20.828616+0000","last_unstale":"2024-03-24T12:10:01.318141+0000","last_undegraded":"2024-03-24T12:10:01.318141+0000","last_fullsized":"2024-03-24T12:10:01.318141+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:14:54.225013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"68'18","reported_seq":61,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317725+0000","last_change":"2024-03-24T12:09:23.145425+0000","last_active":"2024-03-24T12:10:01.317725+0000","last_peered":"2024-03-24T12:10:01.317725+0000","last_clean":"2024-03-24T12:10:01.317725+0000","last_became_active":"2024-03-24T12:09:23.145076+0000","last_became_peered":"2024-03-24T12:09:23.145076+0000","last_unstale":"2024-03-24T12:10:01.317725+0000","last_undegraded":"2024-03-24T12:10:01.317725+0000","last_fullsized":"2024-03-24T12:10:01.317725+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T15:14:42.999283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997048+0000","last_change":"2024-03-24T12:09:25.098716+0000","last_active":"2024-03-24T12:10:00.997048+0000","last_peered":"2024-03-24T12:10:00.997048+0000","last_clean":"2024-03-24T12:10:00.997048+0000","last_became_active":"2024-03-24T12:09:25.098286+0000","last_became_peered":"2024-03-24T12:09:25.098286+0000","last_unstale":"2024-03-24T12:10:00.997048+0000","last_undegraded":"2024-03-24T12:10:00.997048+0000","last_fullsized":"2024-03-24T12:10:00.997048+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:15:47.356055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107605+0000","last_change":"2024-03-24T12:09:27.107290+0000","last_active":"2024-03-24T12:10:01.107605+0000","last_peered":"2024-03-24T12:10:01.107605+0000","last_clean":"2024-03-24T12:10:01.107605+0000","last_became_active":"2024-03-24T12:09:27.106990+0000","last_became_peered":"2024-03-24T12:09:27.106990+0000","last_unstale":"2024-03-24T12:10:01.107605+0000","last_undegraded":"2024-03-24T12:10:01.107605+0000","last_fullsized":"2024-03-24T12:10:01.107605+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T23:05:22.631547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"68'8","reported_seq":49,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:31.500662+0000","last_change":"2024-03-24T12:09:25.098147+0000","last_active":"2024-03-24T12:10:31.500662+0000","last_peered":"2024-03-24T12:10:31.500662+0000","last_clean":"2024-03-24T12:10:31.500662+0000","last_became_active":"2024-03-24T12:09:25.097418+0000","last_became_peered":"2024-03-24T12:09:25.097418+0000","last_unstale":"2024-03-24T12:10:31.500662+0000","last_undegraded":"2024-03-24T12:10:31.500662+0000","last_fullsized":"2024-03-24T12:10:31.500662+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:14:17.359686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317621+0000","last_change":"2024-03-24T12:09:20.827094+0000","last_active":"2024-03-24T12:10:01.317621+0000","last_peered":"2024-03-24T12:10:01.317621+0000","last_clean":"2024-03-24T12:10:01.317621+0000","last_became_active":"2024-03-24T12:09:20.824753+0000","last_became_peered":"2024-03-24T12:09:20.824753+0000","last_unstale":"2024-03-24T12:10:01.317621+0000","last_undegraded":"2024-03-24T12:10:01.317621+0000","last_fullsized":"2024-03-24T12:10:01.317621+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T22:55:10.534182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"68'15","reported_seq":54,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.318418+0000","last_change":"2024-03-24T12:09:23.088102+0000","last_active":"2024-03-24T12:10:01.318418+0000","last_peered":"2024-03-24T12:10:01.318418+0000","last_clean":"2024-03-24T12:10:01.318418+0000","last_became_active":"2024-03-24T12:09:23.087739+0000","last_became_peered":"2024-03-24T12:09:23.087739+0000","last_unstale":"2024-03-24T12:10:01.318418+0000","last_undegraded":"2024-03-24T12:10:01.318418+0000","last_fullsized":"2024-03-24T12:10:01.318418+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:25:35.825369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317574+0000","last_change":"2024-03-24T12:09:27.148982+0000","last_active":"2024-03-24T12:10:01.317574+0000","last_peered":"2024-03-24T12:10:01.317574+0000","last_clean":"2024-03-24T12:10:01.317574+0000","last_became_active":"2024-03-24T12:09:27.148641+0000","last_became_peered":"2024-03-24T12:09:27.148641+0000","last_unstale":"2024-03-24T12:10:01.317574+0000","last_undegraded":"2024-03-24T12:10:01.317574+0000","last_fullsized":"2024-03-24T12:10:01.317574+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:25:21.697003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.108127+0000","last_change":"2024-03-24T12:09:25.099260+0000","last_active":"2024-03-24T12:10:01.108127+0000","last_peered":"2024-03-24T12:10:01.108127+0000","last_clean":"2024-03-24T12:10:01.108127+0000","last_became_active":"2024-03-24T12:09:25.098964+0000","last_became_peered":"2024-03-24T12:09:25.098964+0000","last_unstale":"2024-03-24T12:10:01.108127+0000","last_undegraded":"2024-03-24T12:10:01.108127+0000","last_fullsized":"2024-03-24T12:10:01.108127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T20:46:40.805707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997045+0000","last_change":"2024-03-24T12:09:21.445910+0000","last_active":"2024-03-24T12:10:00.997045+0000","last_peered":"2024-03-24T12:10:00.997045+0000","last_clean":"2024-03-24T12:10:00.997045+0000","last_became_active":"2024-03-24T12:09:21.445619+0000","last_became_peered":"2024-03-24T12:09:21.445619+0000","last_unstale":"2024-03-24T12:10:00.997045+0000","last_undegraded":"2024-03-24T12:10:00.997045+0000","last_fullsized":"2024-03-24T12:10:00.997045+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:54:32.662859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"68'11","reported_seq":48,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.999156+0000","last_change":"2024-03-24T12:09:23.077392+0000","last_active":"2024-03-24T12:10:00.999156+0000","last_peered":"2024-03-24T12:10:00.999156+0000","last_clean":"2024-03-24T12:10:00.999156+0000","last_became_active":"2024-03-24T12:09:23.076962+0000","last_became_peered":"2024-03-24T12:09:23.076962+0000","last_unstale":"2024-03-24T12:10:00.999156+0000","last_undegraded":"2024-03-24T12:10:00.999156+0000","last_fullsized":"2024-03-24T12:10:00.999156+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:15:37.747799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.113290+0000","last_change":"2024-03-24T12:09:27.116400+0000","last_active":"2024-03-24T12:10:01.113290+0000","last_peered":"2024-03-24T12:10:01.113290+0000","last_clean":"2024-03-24T12:10:01.113290+0000","last_became_active":"2024-03-24T12:09:27.116084+0000","last_became_peered":"2024-03-24T12:09:27.116084+0000","last_unstale":"2024-03-24T12:10:01.113290+0000","last_undegraded":"2024-03-24T12:10:01.113290+0000","last_fullsized":"2024-03-24T12:10:01.113290+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T15:36:58.981364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.318430+0000","last_change":"2024-03-24T12:09:25.099499+0000","last_active":"2024-03-24T12:10:01.318430+0000","last_peered":"2024-03-24T12:10:01.318430+0000","last_clean":"2024-03-24T12:10:01.318430+0000","last_became_active":"2024-03-24T12:09:25.099175+0000","last_became_peered":"2024-03-24T12:09:25.099175+0000","last_unstale":"2024-03-24T12:10:01.318430+0000","last_undegraded":"2024-03-24T12:10:01.318430+0000","last_fullsized":"2024-03-24T12:10:01.318430+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T23:26:33.849514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.316958+0000","last_change":"2024-03-24T12:09:20.818832+0000","last_active":"2024-03-24T12:10:01.316958+0000","last_peered":"2024-03-24T12:10:01.316958+0000","last_clean":"2024-03-24T12:10:01.316958+0000","last_became_active":"2024-03-24T12:09:20.818530+0000","last_became_peered":"2024-03-24T12:09:20.818530+0000","last_unstale":"2024-03-24T12:10:01.316958+0000","last_undegraded":"2024-03-24T12:10:01.316958+0000","last_fullsized":"2024-03-24T12:10:01.316958+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:46:08.663561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"68'17","reported_seq":57,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.999665+0000","last_change":"2024-03-24T12:09:23.082282+0000","last_active":"2024-03-24T12:10:00.999665+0000","last_peered":"2024-03-24T12:10:00.999665+0000","last_clean":"2024-03-24T12:10:00.999665+0000","last_became_active":"2024-03-24T12:09:23.081949+0000","last_became_peered":"2024-03-24T12:09:23.081949+0000","last_unstale":"2024-03-24T12:10:00.999665+0000","last_undegraded":"2024-03-24T12:10:00.999665+0000","last_fullsized":"2024-03-24T12:10:00.999665+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:21:38.466360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.999505+0000","last_change":"2024-03-24T12:09:27.111186+0000","last_active":"2024-03-24T12:10:00.999505+0000","last_peered":"2024-03-24T12:10:00.999505+0000","last_clean":"2024-03-24T12:10:00.999505+0000","last_became_active":"2024-03-24T12:09:27.110879+0000","last_became_peered":"2024-03-24T12:09:27.110879+0000","last_unstale":"2024-03-24T12:10:00.999505+0000","last_undegraded":"2024-03-24T12:10:00.999505+0000","last_fullsized":"2024-03-24T12:10:00.999505+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T23:38:00.319682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"68'8","reported_seq":49,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:31.500440+0000","last_change":"2024-03-24T12:09:25.100778+0000","last_active":"2024-03-24T12:10:31.500440+0000","last_peered":"2024-03-24T12:10:31.500440+0000","last_clean":"2024-03-24T12:10:31.500440+0000","last_became_active":"2024-03-24T12:09:25.100474+0000","last_became_peered":"2024-03-24T12:09:25.100474+0000","last_unstale":"2024-03-24T12:10:31.500440+0000","last_undegraded":"2024-03-24T12:10:31.500440+0000","last_fullsized":"2024-03-24T12:10:31.500440+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:34:39.236366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"61'1","reported_seq":39,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317372+0000","last_change":"2024-03-24T12:09:20.820784+0000","last_active":"2024-03-24T12:10:01.317372+0000","last_peered":"2024-03-24T12:10:01.317372+0000","last_clean":"2024-03-24T12:10:01.317372+0000","last_became_active":"2024-03-24T12:09:20.820492+0000","last_became_peered":"2024-03-24T12:09:20.820492+0000","last_unstale":"2024-03-24T12:10:01.317372+0000","last_undegraded":"2024-03-24T12:10:01.317372+0000","last_fullsized":"2024-03-24T12:10:01.317372+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:30:28.025900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"68'10","reported_seq":44,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.998137+0000","last_change":"2024-03-24T12:09:23.084398+0000","last_active":"2024-03-24T12:10:00.998137+0000","last_peered":"2024-03-24T12:10:00.998137+0000","last_clean":"2024-03-24T12:10:00.998137+0000","last_became_active":"2024-03-24T12:09:23.083831+0000","last_became_peered":"2024-03-24T12:09:23.083831+0000","last_unstale":"2024-03-24T12:10:00.998137+0000","last_undegraded":"2024-03-24T12:10:00.998137+0000","last_fullsized":"2024-03-24T12:10:00.998137+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:01:00.297743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":383,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1457,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":64,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234004,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28388,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716764,"statfs":{"total":95995035648,"available":95965966336,"internally_reserved":0,"allocated":1404928,"data_stored":928991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495641,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32532,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712620,"statfs":{"total":95995035648,"available":95961722880,"internally_reserved":0,"allocated":1388544,"data_stored":928814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":839680,"data_stored":339013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28396,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716756,"statfs":{"total":95995035648,"available":95965958144,"internally_reserved":0,"allocated":1413120,"data_stored":929993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31956,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713196,"statfs":{"total":95995035648,"available":95962312704,"internally_reserved":0,"allocated":794624,"data_stored":338003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182441,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":831488,"data_stored":338860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018915,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":823296,"data_stored":339436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757278,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31952,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713200,"statfs":{"total":95995035648,"available":95962316800,"internally_reserved":0,"allocated":790528,"data_stored":337884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-24T12:10:39.085 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-24T12:10:40.334 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:10:40.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:40 smithi006 ceph-mon[30108]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:40.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:40 smithi006 ceph-mon[41734]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:40.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:40 smithi090 ceph-mon[33646]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:42.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:10:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:10:42.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:42 smithi090 ceph-mon[33646]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:42.934 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:42 smithi006 ceph-mon[30108]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:42.934 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:42 smithi006 ceph-mon[41734]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:43.681 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:43 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:10:43.682 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:43 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:10:43.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:43 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:10:44.461 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:10:44 smithi006 systemd[1]: Starting Ceph alertmanager.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:10:44.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.699+0000 7fd796bb3700 1 -- 172.21.15.6:0/961735433 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd790107030 msgr2=0x7fd79010ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:44.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.699+0000 7fd796bb3700 1 --2- 172.21.15.6:0/961735433 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd790107030 0x7fd79010ebb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd788009f90 tx=0x7fd78800b330 comp rx=0 tx=0).stop 2024-03-24T12:10:44.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.700+0000 7fd796bb3700 1 -- 172.21.15.6:0/961735433 shutdown_connections 2024-03-24T12:10:44.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.700+0000 7fd796bb3700 1 --2- 172.21.15.6:0/961735433 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd790107030 0x7fd79010ebb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:44.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.700+0000 7fd796bb3700 1 --2- 172.21.15.6:0/961735433 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd790100810 0x7fd790100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:44.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.700+0000 7fd796bb3700 1 --2- 172.21.15.6:0/961735433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7901011c0 0x7fd790106900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:44.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.700+0000 7fd796bb3700 1 -- 172.21.15.6:0/961735433 >> 172.21.15.6:0/961735433 conn(0x7fd790075eb0 msgr2=0x7fd7900762c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:44.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.700+0000 7fd796bb3700 1 -- 172.21.15.6:0/961735433 shutdown_connections 2024-03-24T12:10:44.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.700+0000 7fd796bb3700 1 -- 172.21.15.6:0/961735433 wait complete. 2024-03-24T12:10:44.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.700+0000 7fd796bb3700 1 Processor -- start 2024-03-24T12:10:44.701 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.700+0000 7fd796bb3700 1 -- start start 2024-03-24T12:10:44.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.700+0000 7fd796bb3700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd790100810 0x7fd790101790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:44.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.700+0000 7fd796bb3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7901011c0 0x7fd790101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:44.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.700+0000 7fd796bb3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd790107030 0x7fd790102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:44.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.701+0000 7fd796bb3700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7901a5c30 con 0x7fd7901011c0 2024-03-24T12:10:44.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.701+0000 7fd796bb3700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7901a5da0 con 0x7fd790107030 2024-03-24T12:10:44.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.701+0000 7fd796bb3700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7901a5f60 con 0x7fd790100810 2024-03-24T12:10:44.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.701+0000 7fd79494f700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd790100810 0x7fd790101790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:44.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.701+0000 7fd79494f700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd790100810 0x7fd790101790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:41768/0 (socket says 172.21.15.6:41768) 2024-03-24T12:10:44.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.701+0000 7fd79494f700 1 -- 172.21.15.6:0/2972686367 learned_addr learned my addr 172.21.15.6:0/2972686367 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:10:44.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.701+0000 7fd78ffff700 1 --2- 172.21.15.6:0/2972686367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7901011c0 0x7fd790101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:44.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.701+0000 7fd795150700 1 --2- 172.21.15.6:0/2972686367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd790107030 0x7fd790102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:44.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.701+0000 7fd79494f700 1 -- 172.21.15.6:0/2972686367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd790107030 msgr2=0x7fd790102210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:44.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.701+0000 7fd79494f700 1 --2- 172.21.15.6:0/2972686367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd790107030 0x7fd790102210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:44.702 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.701+0000 7fd79494f700 1 -- 172.21.15.6:0/2972686367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7901011c0 msgr2=0x7fd790101cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:44.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.701+0000 7fd79494f700 1 --2- 172.21.15.6:0/2972686367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7901011c0 0x7fd790101cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:44.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.701+0000 7fd79494f700 1 -- 172.21.15.6:0/2972686367 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7840095f0 con 0x7fd790100810 2024-03-24T12:10:44.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.701+0000 7fd795150700 1 --2- 172.21.15.6:0/2972686367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd790107030 0x7fd790102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T12:10:44.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.701+0000 7fd79494f700 1 --2- 172.21.15.6:0/2972686367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd790100810 0x7fd790101790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd78400ec10 tx=0x7fd78400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:44.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.701+0000 7fd78dffb700 1 -- 172.21.15.6:0/2972686367 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78400cd40 con 0x7fd790100810 2024-03-24T12:10:44.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.701+0000 7fd796bb3700 1 -- 172.21.15.6:0/2972686367 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd788009c40 con 0x7fd790100810 2024-03-24T12:10:44.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.701+0000 7fd78dffb700 1 -- 172.21.15.6:0/2972686367 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd7840044a0 con 0x7fd790100810 2024-03-24T12:10:44.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.701+0000 7fd78dffb700 1 -- 172.21.15.6:0/2972686367 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784010430 con 0x7fd790100810 2024-03-24T12:10:44.703 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.702+0000 7fd796bb3700 1 -- 172.21.15.6:0/2972686367 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7901a65b0 con 0x7fd790100810 2024-03-24T12:10:44.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.703+0000 7fd796bb3700 1 -- 172.21.15.6:0/2972686367 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd79010a270 con 0x7fd790100810 2024-03-24T12:10:44.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.703+0000 7fd78dffb700 1 -- 172.21.15.6:0/2972686367 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7fd784003f50 con 0x7fd790100810 2024-03-24T12:10:44.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.703+0000 7fd78dffb700 1 --2- 172.21.15.6:0/2972686367 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fd78006d660 0x7fd78006fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:44.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.703+0000 7fd78dffb700 1 -- 172.21.15.6:0/2972686367 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7fd784014070 con 0x7fd790100810 2024-03-24T12:10:44.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.703+0000 7fd78ffff700 1 --2- 172.21.15.6:0/2972686367 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fd78006d660 0x7fd78006fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:44.704 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.704+0000 7fd78ffff700 1 --2- 172.21.15.6:0/2972686367 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fd78006d660 0x7fd78006fb20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd790108440 tx=0x7fd77c00b410 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:44.707 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.708+0000 7fd78dffb700 1 -- 172.21.15.6:0/2972686367 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd784056e10 con 0x7fd790100810 2024-03-24T12:10:44.910 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.910+0000 7fd796bb3700 1 -- 172.21.15.6:0/2972686367 --> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd790102cc0 con 0x7fd78006d660 2024-03-24T12:10:44.910 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:44 smithi006 ceph-mon[30108]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:10:44.911 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:44 smithi006 ceph-mon[41734]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:10:44.919 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.920+0000 7fd78dffb700 1 -- 172.21.15.6:0/2972686367 <== mgr.24464 v2:172.21.15.6:6800/3865852080 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+346476 (secure 0 0 0) 0x7fd790102cc0 con 0x7fd78006d660 2024-03-24T12:10:44.924 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:10:44.926 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.924+0000 7fd796bb3700 1 -- 172.21.15.6:0/2972686367 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fd78006d660 msgr2=0x7fd78006fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:44.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.924+0000 7fd796bb3700 1 --2- 172.21.15.6:0/2972686367 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fd78006d660 0x7fd78006fb20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd790108440 tx=0x7fd77c00b410 comp rx=0 tx=0).stop 2024-03-24T12:10:44.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.924+0000 7fd796bb3700 1 -- 172.21.15.6:0/2972686367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd790100810 msgr2=0x7fd790101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:44.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.924+0000 7fd796bb3700 1 --2- 172.21.15.6:0/2972686367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd790100810 0x7fd790101790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd78400ec10 tx=0x7fd78400ef20 comp rx=0 tx=0).stop 2024-03-24T12:10:44.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.924+0000 7fd796bb3700 1 -- 172.21.15.6:0/2972686367 shutdown_connections 2024-03-24T12:10:44.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.924+0000 7fd796bb3700 1 --2- 172.21.15.6:0/2972686367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd790107030 0x7fd790102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:44.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.924+0000 7fd796bb3700 1 --2- 172.21.15.6:0/2972686367 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fd78006d660 0x7fd78006fb20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:44.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.924+0000 7fd796bb3700 1 --2- 172.21.15.6:0/2972686367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd790100810 0x7fd790101790 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:44.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.924+0000 7fd796bb3700 1 --2- 172.21.15.6:0/2972686367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7901011c0 0x7fd790101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:44.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.924+0000 7fd796bb3700 1 -- 172.21.15.6:0/2972686367 >> 172.21.15.6:0/2972686367 conn(0x7fd790075eb0 msgr2=0x7fd7900fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:44.927 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.925+0000 7fd796bb3700 1 -- 172.21.15.6:0/2972686367 shutdown_connections 2024-03-24T12:10:44.928 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:44.925+0000 7fd796bb3700 1 -- 172.21.15.6:0/2972686367 wait complete. 2024-03-24T12:10:44.928 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2024-03-24T12:10:44.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:44 smithi090 ceph-mon[33646]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:10:45.165 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:10:44 smithi006 podman[79357]: 2024-03-24 12:10:44.9593705 +0000 UTC m=+0.115246113 volume create 37a356bf24e4ca5a3d38cf41b3c6e6a246e203d842760e35179ecb5d65e2e0ea 2024-03-24T12:10:45.165 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:10:44 smithi006 podman[79357]: 2024-03-24 12:10:44.862594292 +0000 UTC m=+0.018469871 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-03-24T12:10:45.165 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:10:45 smithi006 podman[79357]: 2024-03-24 12:10:45.073020514 +0000 UTC m=+0.228896096 container create bba13843ec61747d522d369757ecee450b507e8be0fd8406d985ee0522ede4ae (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T12:10:45.434 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:10:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:10:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:10:45.898 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-03-24T12:10:43.013862+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596318,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1008,"num_read_kb":854,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1771434,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":245040,"kb_used_data":8092,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":749716176,"statfs":{"total":767960285184,"available":767709364224,"internally_reserved":0,"allocated":8286208,"data_stored":4480994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003119"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997070+0000","last_change":"2024-03-24T12:09:27.111366+0000","last_active":"2024-03-24T12:10:00.997070+0000","last_peered":"2024-03-24T12:10:00.997070+0000","last_clean":"2024-03-24T12:10:00.997070+0000","last_became_active":"2024-03-24T12:09:27.110948+0000","last_became_peered":"2024-03-24T12:09:27.110948+0000","last_unstale":"2024-03-24T12:10:00.997070+0000","last_undegraded":"2024-03-24T12:10:00.997070+0000","last_fullsized":"2024-03-24T12:10:00.997070+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:35:32.019484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317036+0000","last_change":"2024-03-24T12:09:25.100258+0000","last_active":"2024-03-24T12:10:01.317036+0000","last_peered":"2024-03-24T12:10:01.317036+0000","last_clean":"2024-03-24T12:10:01.317036+0000","last_became_active":"2024-03-24T12:09:25.099947+0000","last_became_peered":"2024-03-24T12:09:25.099947+0000","last_unstale":"2024-03-24T12:10:01.317036+0000","last_undegraded":"2024-03-24T12:10:01.317036+0000","last_fullsized":"2024-03-24T12:10:01.317036+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:30:16.001334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.995075+0000","last_change":"2024-03-24T12:09:21.445304+0000","last_active":"2024-03-24T12:10:00.995075+0000","last_peered":"2024-03-24T12:10:00.995075+0000","last_clean":"2024-03-24T12:10:00.995075+0000","last_became_active":"2024-03-24T12:09:21.444950+0000","last_became_peered":"2024-03-24T12:09:21.444950+0000","last_unstale":"2024-03-24T12:10:00.995075+0000","last_undegraded":"2024-03-24T12:10:00.995075+0000","last_fullsized":"2024-03-24T12:10:00.995075+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T15:05:05.674601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.996981+0000","last_change":"2024-03-24T12:09:23.077223+0000","last_active":"2024-03-24T12:10:00.996981+0000","last_peered":"2024-03-24T12:10:00.996981+0000","last_clean":"2024-03-24T12:10:00.996981+0000","last_became_active":"2024-03-24T12:09:23.076849+0000","last_became_peered":"2024-03-24T12:09:23.076849+0000","last_unstale":"2024-03-24T12:10:00.996981+0000","last_undegraded":"2024-03-24T12:10:00.996981+0000","last_fullsized":"2024-03-24T12:10:00.996981+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T23:43:37.825165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"68'1","reported_seq":22,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317233+0000","last_change":"2024-03-24T12:09:27.150853+0000","last_active":"2024-03-24T12:10:01.317233+0000","last_peered":"2024-03-24T12:10:01.317233+0000","last_clean":"2024-03-24T12:10:01.317233+0000","last_became_active":"2024-03-24T12:09:27.150496+0000","last_became_peered":"2024-03-24T12:09:27.150496+0000","last_unstale":"2024-03-24T12:10:01.317233+0000","last_undegraded":"2024-03-24T12:10:01.317233+0000","last_fullsized":"2024-03-24T12:10:01.317233+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T23:37:35.131084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.996988+0000","last_change":"2024-03-24T12:09:25.102274+0000","last_active":"2024-03-24T12:10:00.996988+0000","last_peered":"2024-03-24T12:10:00.996988+0000","last_clean":"2024-03-24T12:10:00.996988+0000","last_became_active":"2024-03-24T12:09:25.101995+0000","last_became_peered":"2024-03-24T12:09:25.101995+0000","last_unstale":"2024-03-24T12:10:00.996988+0000","last_undegraded":"2024-03-24T12:10:00.996988+0000","last_fullsized":"2024-03-24T12:10:00.996988+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:19:07.281270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.318722+0000","last_change":"2024-03-24T12:09:21.444675+0000","last_active":"2024-03-24T12:10:01.318722+0000","last_peered":"2024-03-24T12:10:01.318722+0000","last_clean":"2024-03-24T12:10:01.318722+0000","last_became_active":"2024-03-24T12:09:21.444335+0000","last_became_peered":"2024-03-24T12:09:21.444335+0000","last_unstale":"2024-03-24T12:10:01.318722+0000","last_undegraded":"2024-03-24T12:10:01.318722+0000","last_fullsized":"2024-03-24T12:10:01.318722+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:22:11.203735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"68'15","reported_seq":54,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.319019+0000","last_change":"2024-03-24T12:09:23.086498+0000","last_active":"2024-03-24T12:10:01.319019+0000","last_peered":"2024-03-24T12:10:01.319019+0000","last_clean":"2024-03-24T12:10:01.319019+0000","last_became_active":"2024-03-24T12:09:23.086077+0000","last_became_peered":"2024-03-24T12:09:23.086077+0000","last_unstale":"2024-03-24T12:10:01.319019+0000","last_undegraded":"2024-03-24T12:10:01.319019+0000","last_fullsized":"2024-03-24T12:10:01.319019+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T22:52:20.448715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.994198+0000","last_change":"2024-03-24T12:09:27.120441+0000","last_active":"2024-03-24T12:10:00.994198+0000","last_peered":"2024-03-24T12:10:00.994198+0000","last_clean":"2024-03-24T12:10:00.994198+0000","last_became_active":"2024-03-24T12:09:27.120132+0000","last_became_peered":"2024-03-24T12:09:27.120132+0000","last_unstale":"2024-03-24T12:10:00.994198+0000","last_undegraded":"2024-03-24T12:10:00.994198+0000","last_fullsized":"2024-03-24T12:10:00.994198+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:16:03.656763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107490+0000","last_change":"2024-03-24T12:09:25.106706+0000","last_active":"2024-03-24T12:10:01.107490+0000","last_peered":"2024-03-24T12:10:01.107490+0000","last_clean":"2024-03-24T12:10:01.107490+0000","last_became_active":"2024-03-24T12:09:25.106225+0000","last_became_peered":"2024-03-24T12:09:25.106225+0000","last_unstale":"2024-03-24T12:10:01.107490+0000","last_undegraded":"2024-03-24T12:10:01.107490+0000","last_fullsized":"2024-03-24T12:10:01.107490+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:11:02.213531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107911+0000","last_change":"2024-03-24T12:09:20.824379+0000","last_active":"2024-03-24T12:10:01.107911+0000","last_peered":"2024-03-24T12:10:01.107911+0000","last_clean":"2024-03-24T12:10:01.107911+0000","last_became_active":"2024-03-24T12:09:20.823417+0000","last_became_peered":"2024-03-24T12:09:20.823417+0000","last_unstale":"2024-03-24T12:10:01.107911+0000","last_undegraded":"2024-03-24T12:10:01.107911+0000","last_fullsized":"2024-03-24T12:10:01.107911+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:35:05.153460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997256+0000","last_change":"2024-03-24T12:09:23.084076+0000","last_active":"2024-03-24T12:10:00.997256+0000","last_peered":"2024-03-24T12:10:00.997256+0000","last_clean":"2024-03-24T12:10:00.997256+0000","last_became_active":"2024-03-24T12:09:23.083717+0000","last_became_peered":"2024-03-24T12:09:23.083717+0000","last_unstale":"2024-03-24T12:10:00.997256+0000","last_undegraded":"2024-03-24T12:10:00.997256+0000","last_fullsized":"2024-03-24T12:10:00.997256+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:13:19.903943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107309+0000","last_change":"2024-03-24T12:09:27.118024+0000","last_active":"2024-03-24T12:10:01.107309+0000","last_peered":"2024-03-24T12:10:01.107309+0000","last_clean":"2024-03-24T12:10:01.107309+0000","last_became_active":"2024-03-24T12:09:27.117712+0000","last_became_peered":"2024-03-24T12:09:27.117712+0000","last_unstale":"2024-03-24T12:10:01.107309+0000","last_undegraded":"2024-03-24T12:10:01.107309+0000","last_fullsized":"2024-03-24T12:10:01.107309+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:42:22.383046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.115936+0000","last_change":"2024-03-24T12:09:25.105091+0000","last_active":"2024-03-24T12:10:01.115936+0000","last_peered":"2024-03-24T12:10:01.115936+0000","last_clean":"2024-03-24T12:10:01.115936+0000","last_became_active":"2024-03-24T12:09:25.104771+0000","last_became_peered":"2024-03-24T12:09:25.104771+0000","last_unstale":"2024-03-24T12:10:01.115936+0000","last_undegraded":"2024-03-24T12:10:01.115936+0000","last_fullsized":"2024-03-24T12:10:01.115936+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:27:28.765558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107558+0000","last_change":"2024-03-24T12:09:20.821356+0000","last_active":"2024-03-24T12:10:01.107558+0000","last_peered":"2024-03-24T12:10:01.107558+0000","last_clean":"2024-03-24T12:10:01.107558+0000","last_became_active":"2024-03-24T12:09:20.821052+0000","last_became_peered":"2024-03-24T12:09:20.821052+0000","last_unstale":"2024-03-24T12:10:01.107558+0000","last_undegraded":"2024-03-24T12:10:01.107558+0000","last_fullsized":"2024-03-24T12:10:01.107558+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:18:27.681064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"68'5","reported_seq":39,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997854+0000","last_change":"2024-03-24T12:09:23.084755+0000","last_active":"2024-03-24T12:10:00.997854+0000","last_peered":"2024-03-24T12:10:00.997854+0000","last_clean":"2024-03-24T12:10:00.997854+0000","last_became_active":"2024-03-24T12:09:23.084425+0000","last_became_peered":"2024-03-24T12:09:23.084425+0000","last_unstale":"2024-03-24T12:10:00.997854+0000","last_undegraded":"2024-03-24T12:10:00.997854+0000","last_fullsized":"2024-03-24T12:10:00.997854+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:14:43.375057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.999291+0000","last_change":"2024-03-24T12:09:27.151256+0000","last_active":"2024-03-24T12:10:00.999291+0000","last_peered":"2024-03-24T12:10:00.999291+0000","last_clean":"2024-03-24T12:10:00.999291+0000","last_became_active":"2024-03-24T12:09:27.150904+0000","last_became_peered":"2024-03-24T12:09:27.150904+0000","last_unstale":"2024-03-24T12:10:00.999291+0000","last_undegraded":"2024-03-24T12:10:00.999291+0000","last_fullsized":"2024-03-24T12:10:00.999291+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:16:44.043383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317777+0000","last_change":"2024-03-24T12:09:25.099346+0000","last_active":"2024-03-24T12:10:01.317777+0000","last_peered":"2024-03-24T12:10:01.317777+0000","last_clean":"2024-03-24T12:10:01.317777+0000","last_became_active":"2024-03-24T12:09:25.099032+0000","last_became_peered":"2024-03-24T12:09:25.099032+0000","last_unstale":"2024-03-24T12:10:01.317777+0000","last_undegraded":"2024-03-24T12:10:01.317777+0000","last_fullsized":"2024-03-24T12:10:01.317777+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:49:13.644904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.995223+0000","last_change":"2024-03-24T12:09:20.827565+0000","last_active":"2024-03-24T12:10:00.995223+0000","last_peered":"2024-03-24T12:10:00.995223+0000","last_clean":"2024-03-24T12:10:00.995223+0000","last_became_active":"2024-03-24T12:09:20.827251+0000","last_became_peered":"2024-03-24T12:09:20.827251+0000","last_unstale":"2024-03-24T12:10:00.995223+0000","last_undegraded":"2024-03-24T12:10:00.995223+0000","last_fullsized":"2024-03-24T12:10:00.995223+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:15:57.815587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"68'15","reported_seq":54,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.113419+0000","last_change":"2024-03-24T12:09:23.088416+0000","last_active":"2024-03-24T12:10:01.113419+0000","last_peered":"2024-03-24T12:10:01.113419+0000","last_clean":"2024-03-24T12:10:01.113419+0000","last_became_active":"2024-03-24T12:09:23.088067+0000","last_became_peered":"2024-03-24T12:09:23.088067+0000","last_unstale":"2024-03-24T12:10:01.113419+0000","last_undegraded":"2024-03-24T12:10:01.113419+0000","last_fullsized":"2024-03-24T12:10:01.113419+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T15:03:27.932435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.318081+0000","last_change":"2024-03-24T12:09:27.150904+0000","last_active":"2024-03-24T12:10:01.318081+0000","last_peered":"2024-03-24T12:10:01.318081+0000","last_clean":"2024-03-24T12:10:01.318081+0000","last_became_active":"2024-03-24T12:09:27.150486+0000","last_became_peered":"2024-03-24T12:09:27.150486+0000","last_unstale":"2024-03-24T12:10:01.318081+0000","last_undegraded":"2024-03-24T12:10:01.318081+0000","last_fullsized":"2024-03-24T12:10:01.318081+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T23:30:35.024129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997732+0000","last_change":"2024-03-24T12:09:25.098180+0000","last_active":"2024-03-24T12:10:00.997732+0000","last_peered":"2024-03-24T12:10:00.997732+0000","last_clean":"2024-03-24T12:10:00.997732+0000","last_became_active":"2024-03-24T12:09:25.097874+0000","last_became_peered":"2024-03-24T12:09:25.097874+0000","last_unstale":"2024-03-24T12:10:00.997732+0000","last_undegraded":"2024-03-24T12:10:00.997732+0000","last_fullsized":"2024-03-24T12:10:00.997732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:03:12.128447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997409+0000","last_change":"2024-03-24T12:09:21.444812+0000","last_active":"2024-03-24T12:10:00.997409+0000","last_peered":"2024-03-24T12:10:00.997409+0000","last_clean":"2024-03-24T12:10:00.997409+0000","last_became_active":"2024-03-24T12:09:21.444497+0000","last_became_peered":"2024-03-24T12:09:21.444497+0000","last_unstale":"2024-03-24T12:10:00.997409+0000","last_undegraded":"2024-03-24T12:10:00.997409+0000","last_fullsized":"2024-03-24T12:10:00.997409+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:50:59.952567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"68'12","reported_seq":52,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317798+0000","last_change":"2024-03-24T12:09:23.085451+0000","last_active":"2024-03-24T12:10:01.317798+0000","last_peered":"2024-03-24T12:10:01.317798+0000","last_clean":"2024-03-24T12:10:01.317798+0000","last_became_active":"2024-03-24T12:09:23.085117+0000","last_became_peered":"2024-03-24T12:09:23.085117+0000","last_unstale":"2024-03-24T12:10:01.317798+0000","last_undegraded":"2024-03-24T12:10:01.317798+0000","last_fullsized":"2024-03-24T12:10:01.317798+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:41:28.199959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"68'1","reported_seq":22,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.116290+0000","last_change":"2024-03-24T12:09:27.114268+0000","last_active":"2024-03-24T12:10:01.116290+0000","last_peered":"2024-03-24T12:10:01.116290+0000","last_clean":"2024-03-24T12:10:01.116290+0000","last_became_active":"2024-03-24T12:09:27.114007+0000","last_became_peered":"2024-03-24T12:09:27.114007+0000","last_unstale":"2024-03-24T12:10:01.116290+0000","last_undegraded":"2024-03-24T12:10:01.116290+0000","last_fullsized":"2024-03-24T12:10:01.116290+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:00:20.199850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":370,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"68'8","reported_seq":53,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:36.501645+0000","last_change":"2024-03-24T12:09:25.095318+0000","last_active":"2024-03-24T12:10:36.501645+0000","last_peered":"2024-03-24T12:10:36.501645+0000","last_clean":"2024-03-24T12:10:36.501645+0000","last_became_active":"2024-03-24T12:09:25.093987+0000","last_became_peered":"2024-03-24T12:09:25.093987+0000","last_unstale":"2024-03-24T12:10:36.501645+0000","last_undegraded":"2024-03-24T12:10:36.501645+0000","last_fullsized":"2024-03-24T12:10:36.501645+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:33:56.774618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"61'1","reported_seq":34,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317132+0000","last_change":"2024-03-24T12:09:20.820519+0000","last_active":"2024-03-24T12:10:01.317132+0000","last_peered":"2024-03-24T12:10:01.317132+0000","last_clean":"2024-03-24T12:10:01.317132+0000","last_became_active":"2024-03-24T12:09:20.820159+0000","last_became_peered":"2024-03-24T12:09:20.820159+0000","last_unstale":"2024-03-24T12:10:01.317132+0000","last_undegraded":"2024-03-24T12:10:01.317132+0000","last_fullsized":"2024-03-24T12:10:01.317132+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T22:47:19.125795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"68'10","reported_seq":44,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.994921+0000","last_change":"2024-03-24T12:09:23.145251+0000","last_active":"2024-03-24T12:10:00.994921+0000","last_peered":"2024-03-24T12:10:00.994921+0000","last_clean":"2024-03-24T12:10:00.994921+0000","last_became_active":"2024-03-24T12:09:23.144937+0000","last_became_peered":"2024-03-24T12:09:23.144937+0000","last_unstale":"2024-03-24T12:10:00.994921+0000","last_undegraded":"2024-03-24T12:10:00.994921+0000","last_fullsized":"2024-03-24T12:10:00.994921+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-26T00:05:00.172163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317487+0000","last_change":"2024-03-24T12:09:27.109797+0000","last_active":"2024-03-24T12:10:01.317487+0000","last_peered":"2024-03-24T12:10:01.317487+0000","last_clean":"2024-03-24T12:10:01.317487+0000","last_became_active":"2024-03-24T12:09:27.109445+0000","last_became_peered":"2024-03-24T12:09:27.109445+0000","last_unstale":"2024-03-24T12:10:01.317487+0000","last_undegraded":"2024-03-24T12:10:01.317487+0000","last_fullsized":"2024-03-24T12:10:01.317487+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:28:47.200374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.995756+0000","last_change":"2024-03-24T12:09:25.095275+0000","last_active":"2024-03-24T12:10:00.995756+0000","last_peered":"2024-03-24T12:10:00.995756+0000","last_clean":"2024-03-24T12:10:00.995756+0000","last_became_active":"2024-03-24T12:09:25.094721+0000","last_became_peered":"2024-03-24T12:09:25.094721+0000","last_unstale":"2024-03-24T12:10:00.995756+0000","last_undegraded":"2024-03-24T12:10:00.995756+0000","last_fullsized":"2024-03-24T12:10:00.995756+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:11:59.980171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317420+0000","last_change":"2024-03-24T12:09:20.823426+0000","last_active":"2024-03-24T12:10:01.317420+0000","last_peered":"2024-03-24T12:10:01.317420+0000","last_clean":"2024-03-24T12:10:01.317420+0000","last_became_active":"2024-03-24T12:09:20.823110+0000","last_became_peered":"2024-03-24T12:09:20.823110+0000","last_unstale":"2024-03-24T12:10:01.317420+0000","last_undegraded":"2024-03-24T12:10:01.317420+0000","last_fullsized":"2024-03-24T12:10:01.317420+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T20:12:28.514808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"68'11","reported_seq":66,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317220+0000","last_change":"2024-03-24T12:09:46.976121+0000","last_active":"2024-03-24T12:10:01.317220+0000","last_peered":"2024-03-24T12:10:01.317220+0000","last_clean":"2024-03-24T12:10:01.317220+0000","last_became_active":"2024-03-24T12:09:46.963045+0000","last_became_peered":"2024-03-24T12:09:46.963045+0000","last_unstale":"2024-03-24T12:10:01.317220+0000","last_undegraded":"2024-03-24T12:10:01.317220+0000","last_fullsized":"2024-03-24T12:10:01.317220+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:39:07.103927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.115933+0000","last_change":"2024-03-24T12:09:27.111424+0000","last_active":"2024-03-24T12:10:01.115933+0000","last_peered":"2024-03-24T12:10:01.115933+0000","last_clean":"2024-03-24T12:10:01.115933+0000","last_became_active":"2024-03-24T12:09:27.111137+0000","last_became_peered":"2024-03-24T12:09:27.111137+0000","last_unstale":"2024-03-24T12:10:01.115933+0000","last_undegraded":"2024-03-24T12:10:01.115933+0000","last_fullsized":"2024-03-24T12:10:01.115933+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:23:12.613402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.316698+0000","last_change":"2024-03-24T12:09:25.094396+0000","last_active":"2024-03-24T12:10:01.316698+0000","last_peered":"2024-03-24T12:10:01.316698+0000","last_clean":"2024-03-24T12:10:01.316698+0000","last_became_active":"2024-03-24T12:09:25.094125+0000","last_became_peered":"2024-03-24T12:09:25.094125+0000","last_unstale":"2024-03-24T12:10:01.316698+0000","last_undegraded":"2024-03-24T12:10:01.316698+0000","last_fullsized":"2024-03-24T12:10:01.316698+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T22:59:17.597978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.995239+0000","last_change":"2024-03-24T12:09:20.825947+0000","last_active":"2024-03-24T12:10:00.995239+0000","last_peered":"2024-03-24T12:10:00.995239+0000","last_clean":"2024-03-24T12:10:00.995239+0000","last_became_active":"2024-03-24T12:09:20.825525+0000","last_became_peered":"2024-03-24T12:09:20.825525+0000","last_unstale":"2024-03-24T12:10:00.995239+0000","last_undegraded":"2024-03-24T12:10:00.995239+0000","last_fullsized":"2024-03-24T12:10:00.995239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T15:29:58.677145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"68'4","reported_seq":35,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317513+0000","last_change":"2024-03-24T12:09:23.084690+0000","last_active":"2024-03-24T12:10:01.317513+0000","last_peered":"2024-03-24T12:10:01.317513+0000","last_clean":"2024-03-24T12:10:01.317513+0000","last_became_active":"2024-03-24T12:09:23.084372+0000","last_became_peered":"2024-03-24T12:09:23.084372+0000","last_unstale":"2024-03-24T12:10:01.317513+0000","last_undegraded":"2024-03-24T12:10:01.317513+0000","last_fullsized":"2024-03-24T12:10:01.317513+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:44:24.541196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317025+0000","last_change":"2024-03-24T12:09:27.149152+0000","last_active":"2024-03-24T12:10:01.317025+0000","last_peered":"2024-03-24T12:10:01.317025+0000","last_clean":"2024-03-24T12:10:01.317025+0000","last_became_active":"2024-03-24T12:09:27.148824+0000","last_became_peered":"2024-03-24T12:09:27.148824+0000","last_unstale":"2024-03-24T12:10:01.317025+0000","last_undegraded":"2024-03-24T12:10:01.317025+0000","last_fullsized":"2024-03-24T12:10:01.317025+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:41:04.422011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.116251+0000","last_change":"2024-03-24T12:09:25.103657+0000","last_active":"2024-03-24T12:10:01.116251+0000","last_peered":"2024-03-24T12:10:01.116251+0000","last_clean":"2024-03-24T12:10:01.116251+0000","last_became_active":"2024-03-24T12:09:25.103201+0000","last_became_peered":"2024-03-24T12:09:25.103201+0000","last_unstale":"2024-03-24T12:10:01.116251+0000","last_undegraded":"2024-03-24T12:10:01.116251+0000","last_fullsized":"2024-03-24T12:10:01.116251+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:21:15.682551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107989+0000","last_change":"2024-03-24T12:09:20.824532+0000","last_active":"2024-03-24T12:10:01.107989+0000","last_peered":"2024-03-24T12:10:01.107989+0000","last_clean":"2024-03-24T12:10:01.107989+0000","last_became_active":"2024-03-24T12:09:20.824171+0000","last_became_peered":"2024-03-24T12:09:20.824171+0000","last_unstale":"2024-03-24T12:10:01.107989+0000","last_undegraded":"2024-03-24T12:10:01.107989+0000","last_fullsized":"2024-03-24T12:10:01.107989+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T23:51:17.104336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"68'11","reported_seq":48,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.319023+0000","last_change":"2024-03-24T12:09:23.143989+0000","last_active":"2024-03-24T12:10:01.319023+0000","last_peered":"2024-03-24T12:10:01.319023+0000","last_clean":"2024-03-24T12:10:01.319023+0000","last_became_active":"2024-03-24T12:09:23.143603+0000","last_became_peered":"2024-03-24T12:09:23.143603+0000","last_unstale":"2024-03-24T12:10:01.319023+0000","last_undegraded":"2024-03-24T12:10:01.319023+0000","last_fullsized":"2024-03-24T12:10:01.319023+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:49:59.464908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.993863+0000","last_change":"2024-03-24T12:09:27.120530+0000","last_active":"2024-03-24T12:10:00.993863+0000","last_peered":"2024-03-24T12:10:00.993863+0000","last_clean":"2024-03-24T12:10:00.993863+0000","last_became_active":"2024-03-24T12:09:27.120213+0000","last_became_peered":"2024-03-24T12:09:27.120213+0000","last_unstale":"2024-03-24T12:10:00.993863+0000","last_undegraded":"2024-03-24T12:10:00.993863+0000","last_fullsized":"2024-03-24T12:10:00.993863+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T23:24:49.314839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317350+0000","last_change":"2024-03-24T12:09:25.103830+0000","last_active":"2024-03-24T12:10:01.317350+0000","last_peered":"2024-03-24T12:10:01.317350+0000","last_clean":"2024-03-24T12:10:01.317350+0000","last_became_active":"2024-03-24T12:09:25.103516+0000","last_became_peered":"2024-03-24T12:09:25.103516+0000","last_unstale":"2024-03-24T12:10:01.317350+0000","last_undegraded":"2024-03-24T12:10:01.317350+0000","last_fullsized":"2024-03-24T12:10:01.317350+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T22:07:14.468870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.116405+0000","last_change":"2024-03-24T12:09:20.825215+0000","last_active":"2024-03-24T12:10:01.116405+0000","last_peered":"2024-03-24T12:10:01.116405+0000","last_clean":"2024-03-24T12:10:01.116405+0000","last_became_active":"2024-03-24T12:09:20.824899+0000","last_became_peered":"2024-03-24T12:09:20.824899+0000","last_unstale":"2024-03-24T12:10:01.116405+0000","last_undegraded":"2024-03-24T12:10:01.116405+0000","last_fullsized":"2024-03-24T12:10:01.116405+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:51:12.685487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317710+0000","last_change":"2024-03-24T12:09:23.086663+0000","last_active":"2024-03-24T12:10:01.317710+0000","last_peered":"2024-03-24T12:10:01.317710+0000","last_clean":"2024-03-24T12:10:01.317710+0000","last_became_active":"2024-03-24T12:09:23.086344+0000","last_became_peered":"2024-03-24T12:09:23.086344+0000","last_unstale":"2024-03-24T12:10:01.317710+0000","last_undegraded":"2024-03-24T12:10:01.317710+0000","last_fullsized":"2024-03-24T12:10:01.317710+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:39:45.014884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.318770+0000","last_change":"2024-03-24T12:09:27.117111+0000","last_active":"2024-03-24T12:10:01.318770+0000","last_peered":"2024-03-24T12:10:01.318770+0000","last_clean":"2024-03-24T12:10:01.318770+0000","last_became_active":"2024-03-24T12:09:27.116792+0000","last_became_peered":"2024-03-24T12:09:27.116792+0000","last_unstale":"2024-03-24T12:10:01.318770+0000","last_undegraded":"2024-03-24T12:10:01.318770+0000","last_fullsized":"2024-03-24T12:10:01.318770+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:34:42.154278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317619+0000","last_change":"2024-03-24T12:09:25.100853+0000","last_active":"2024-03-24T12:10:01.317619+0000","last_peered":"2024-03-24T12:10:01.317619+0000","last_clean":"2024-03-24T12:10:01.317619+0000","last_became_active":"2024-03-24T12:09:25.098585+0000","last_became_peered":"2024-03-24T12:09:25.098585+0000","last_unstale":"2024-03-24T12:10:01.317619+0000","last_undegraded":"2024-03-24T12:10:01.317619+0000","last_fullsized":"2024-03-24T12:10:01.317619+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:53:07.278770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.998323+0000","last_change":"2024-03-24T12:09:20.826612+0000","last_active":"2024-03-24T12:10:00.998323+0000","last_peered":"2024-03-24T12:10:00.998323+0000","last_clean":"2024-03-24T12:10:00.998323+0000","last_became_active":"2024-03-24T12:09:20.826270+0000","last_became_peered":"2024-03-24T12:09:20.826270+0000","last_unstale":"2024-03-24T12:10:00.998323+0000","last_undegraded":"2024-03-24T12:10:00.998323+0000","last_fullsized":"2024-03-24T12:10:00.998323+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:22:08.311526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"68'11","reported_seq":48,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997926+0000","last_change":"2024-03-24T12:09:23.078680+0000","last_active":"2024-03-24T12:10:00.997926+0000","last_peered":"2024-03-24T12:10:00.997926+0000","last_clean":"2024-03-24T12:10:00.997926+0000","last_became_active":"2024-03-24T12:09:23.077788+0000","last_became_peered":"2024-03-24T12:09:23.077788+0000","last_unstale":"2024-03-24T12:10:00.997926+0000","last_undegraded":"2024-03-24T12:10:00.997926+0000","last_fullsized":"2024-03-24T12:10:00.997926+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:49:48.290082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.994580+0000","last_change":"2024-03-24T12:09:27.115226+0000","last_active":"2024-03-24T12:10:00.994580+0000","last_peered":"2024-03-24T12:10:00.994580+0000","last_clean":"2024-03-24T12:10:00.994580+0000","last_became_active":"2024-03-24T12:09:27.114430+0000","last_became_peered":"2024-03-24T12:09:27.114430+0000","last_unstale":"2024-03-24T12:10:00.994580+0000","last_undegraded":"2024-03-24T12:10:00.994580+0000","last_fullsized":"2024-03-24T12:10:00.994580+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:22:41.665035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"68'8","reported_seq":50,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:36.501654+0000","last_change":"2024-03-24T12:09:25.105566+0000","last_active":"2024-03-24T12:10:36.501654+0000","last_peered":"2024-03-24T12:10:36.501654+0000","last_clean":"2024-03-24T12:10:36.501654+0000","last_became_active":"2024-03-24T12:09:25.105262+0000","last_became_peered":"2024-03-24T12:09:25.105262+0000","last_unstale":"2024-03-24T12:10:36.501654+0000","last_undegraded":"2024-03-24T12:10:36.501654+0000","last_fullsized":"2024-03-24T12:10:36.501654+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:17:06.225373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.116023+0000","last_change":"2024-03-24T12:09:21.444535+0000","last_active":"2024-03-24T12:10:01.116023+0000","last_peered":"2024-03-24T12:10:01.116023+0000","last_clean":"2024-03-24T12:10:01.116023+0000","last_became_active":"2024-03-24T12:09:21.444083+0000","last_became_peered":"2024-03-24T12:09:21.444083+0000","last_unstale":"2024-03-24T12:10:01.116023+0000","last_undegraded":"2024-03-24T12:10:01.116023+0000","last_fullsized":"2024-03-24T12:10:01.116023+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T20:35:06.389620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"68'10","reported_seq":44,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317980+0000","last_change":"2024-03-24T12:09:23.144171+0000","last_active":"2024-03-24T12:10:01.317980+0000","last_peered":"2024-03-24T12:10:01.317980+0000","last_clean":"2024-03-24T12:10:01.317980+0000","last_became_active":"2024-03-24T12:09:23.143744+0000","last_became_peered":"2024-03-24T12:09:23.143744+0000","last_unstale":"2024-03-24T12:10:01.317980+0000","last_undegraded":"2024-03-24T12:10:01.317980+0000","last_fullsized":"2024-03-24T12:10:01.317980+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:31:27.373390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.998801+0000","last_change":"2024-03-24T12:09:27.112487+0000","last_active":"2024-03-24T12:10:00.998801+0000","last_peered":"2024-03-24T12:10:00.998801+0000","last_clean":"2024-03-24T12:10:00.998801+0000","last_became_active":"2024-03-24T12:09:27.112179+0000","last_became_peered":"2024-03-24T12:09:27.112179+0000","last_unstale":"2024-03-24T12:10:00.998801+0000","last_undegraded":"2024-03-24T12:10:00.998801+0000","last_fullsized":"2024-03-24T12:10:00.998801+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T20:25:10.244533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"68'8","reported_seq":53,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:36.501776+0000","last_change":"2024-03-24T12:09:25.100165+0000","last_active":"2024-03-24T12:10:36.501776+0000","last_peered":"2024-03-24T12:10:36.501776+0000","last_clean":"2024-03-24T12:10:36.501776+0000","last_became_active":"2024-03-24T12:09:25.099826+0000","last_became_peered":"2024-03-24T12:09:25.099826+0000","last_unstale":"2024-03-24T12:10:36.501776+0000","last_undegraded":"2024-03-24T12:10:36.501776+0000","last_fullsized":"2024-03-24T12:10:36.501776+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T22:45:53.795860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.994195+0000","last_change":"2024-03-24T12:09:20.827524+0000","last_active":"2024-03-24T12:10:00.994195+0000","last_peered":"2024-03-24T12:10:00.994195+0000","last_clean":"2024-03-24T12:10:00.994195+0000","last_became_active":"2024-03-24T12:09:20.827209+0000","last_became_peered":"2024-03-24T12:09:20.827209+0000","last_unstale":"2024-03-24T12:10:00.994195+0000","last_undegraded":"2024-03-24T12:10:00.994195+0000","last_fullsized":"2024-03-24T12:10:00.994195+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:20:53.400737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.108014+0000","last_change":"2024-03-24T12:09:23.147034+0000","last_active":"2024-03-24T12:10:01.108014+0000","last_peered":"2024-03-24T12:10:01.108014+0000","last_clean":"2024-03-24T12:10:01.108014+0000","last_became_active":"2024-03-24T12:09:23.146646+0000","last_became_peered":"2024-03-24T12:09:23.146646+0000","last_unstale":"2024-03-24T12:10:01.108014+0000","last_undegraded":"2024-03-24T12:10:01.108014+0000","last_fullsized":"2024-03-24T12:10:01.108014+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:15:31.126526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.113629+0000","last_change":"2024-03-24T12:09:27.112023+0000","last_active":"2024-03-24T12:10:01.113629+0000","last_peered":"2024-03-24T12:10:01.113629+0000","last_clean":"2024-03-24T12:10:01.113629+0000","last_became_active":"2024-03-24T12:09:27.111576+0000","last_became_peered":"2024-03-24T12:09:27.111576+0000","last_unstale":"2024-03-24T12:10:01.113629+0000","last_undegraded":"2024-03-24T12:10:01.113629+0000","last_fullsized":"2024-03-24T12:10:01.113629+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:54:44.964282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.318248+0000","last_change":"2024-03-24T12:09:25.106357+0000","last_active":"2024-03-24T12:10:01.318248+0000","last_peered":"2024-03-24T12:10:01.318248+0000","last_clean":"2024-03-24T12:10:01.318248+0000","last_became_active":"2024-03-24T12:09:25.106002+0000","last_became_peered":"2024-03-24T12:09:25.106002+0000","last_unstale":"2024-03-24T12:10:01.318248+0000","last_undegraded":"2024-03-24T12:10:01.318248+0000","last_fullsized":"2024-03-24T12:10:01.318248+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:12:42.285452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.115958+0000","last_change":"2024-03-24T12:09:20.826435+0000","last_active":"2024-03-24T12:10:01.115958+0000","last_peered":"2024-03-24T12:10:01.115958+0000","last_clean":"2024-03-24T12:10:01.115958+0000","last_became_active":"2024-03-24T12:09:20.826156+0000","last_became_peered":"2024-03-24T12:09:20.826156+0000","last_unstale":"2024-03-24T12:10:01.115958+0000","last_undegraded":"2024-03-24T12:10:01.115958+0000","last_fullsized":"2024-03-24T12:10:01.115958+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T22:45:03.024321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.995488+0000","last_change":"2024-03-24T12:09:23.144571+0000","last_active":"2024-03-24T12:10:00.995488+0000","last_peered":"2024-03-24T12:10:00.995488+0000","last_clean":"2024-03-24T12:10:00.995488+0000","last_became_active":"2024-03-24T12:09:23.144231+0000","last_became_peered":"2024-03-24T12:09:23.144231+0000","last_unstale":"2024-03-24T12:10:00.995488+0000","last_undegraded":"2024-03-24T12:10:00.995488+0000","last_fullsized":"2024-03-24T12:10:00.995488+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:33:42.836952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.116342+0000","last_change":"2024-03-24T12:09:27.149704+0000","last_active":"2024-03-24T12:10:01.116342+0000","last_peered":"2024-03-24T12:10:01.116342+0000","last_clean":"2024-03-24T12:10:01.116342+0000","last_became_active":"2024-03-24T12:09:27.149378+0000","last_became_peered":"2024-03-24T12:09:27.149378+0000","last_unstale":"2024-03-24T12:10:01.116342+0000","last_undegraded":"2024-03-24T12:10:01.116342+0000","last_fullsized":"2024-03-24T12:10:01.116342+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:42:40.151707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107981+0000","last_change":"2024-03-24T12:09:25.102940+0000","last_active":"2024-03-24T12:10:01.107981+0000","last_peered":"2024-03-24T12:10:01.107981+0000","last_clean":"2024-03-24T12:10:01.107981+0000","last_became_active":"2024-03-24T12:09:25.102625+0000","last_became_peered":"2024-03-24T12:09:25.102625+0000","last_unstale":"2024-03-24T12:10:01.107981+0000","last_undegraded":"2024-03-24T12:10:01.107981+0000","last_fullsized":"2024-03-24T12:10:01.107981+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:00:23.241037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.316645+0000","last_change":"2024-03-24T12:09:20.821247+0000","last_active":"2024-03-24T12:10:01.316645+0000","last_peered":"2024-03-24T12:10:01.316645+0000","last_clean":"2024-03-24T12:10:01.316645+0000","last_became_active":"2024-03-24T12:09:20.820941+0000","last_became_peered":"2024-03-24T12:09:20.820941+0000","last_unstale":"2024-03-24T12:10:01.316645+0000","last_undegraded":"2024-03-24T12:10:01.316645+0000","last_fullsized":"2024-03-24T12:10:01.316645+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:34:38.848780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"68'6","reported_seq":38,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.318690+0000","last_change":"2024-03-24T12:09:23.145277+0000","last_active":"2024-03-24T12:10:01.318690+0000","last_peered":"2024-03-24T12:10:01.318690+0000","last_clean":"2024-03-24T12:10:01.318690+0000","last_became_active":"2024-03-24T12:09:23.144971+0000","last_became_peered":"2024-03-24T12:09:23.144971+0000","last_unstale":"2024-03-24T12:10:01.318690+0000","last_undegraded":"2024-03-24T12:10:01.318690+0000","last_fullsized":"2024-03-24T12:10:01.318690+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T15:54:44.221081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107759+0000","last_change":"2024-03-24T12:09:27.150904+0000","last_active":"2024-03-24T12:10:01.107759+0000","last_peered":"2024-03-24T12:10:01.107759+0000","last_clean":"2024-03-24T12:10:01.107759+0000","last_became_active":"2024-03-24T12:09:27.150531+0000","last_became_peered":"2024-03-24T12:09:27.150531+0000","last_unstale":"2024-03-24T12:10:01.107759+0000","last_undegraded":"2024-03-24T12:10:01.107759+0000","last_fullsized":"2024-03-24T12:10:01.107759+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:36:29.239350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"61'2","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.116682+0000","last_change":"2024-03-24T12:09:20.827574+0000","last_active":"2024-03-24T12:10:01.116682+0000","last_peered":"2024-03-24T12:10:01.116682+0000","last_clean":"2024-03-24T12:10:01.116682+0000","last_became_active":"2024-03-24T12:09:20.827179+0000","last_became_peered":"2024-03-24T12:09:20.827179+0000","last_unstale":"2024-03-24T12:10:01.116682+0000","last_undegraded":"2024-03-24T12:10:01.116682+0000","last_fullsized":"2024-03-24T12:10:01.116682+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:11:47.157188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"68'15","reported_seq":54,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107293+0000","last_change":"2024-03-24T12:09:23.145502+0000","last_active":"2024-03-24T12:10:01.107293+0000","last_peered":"2024-03-24T12:10:01.107293+0000","last_clean":"2024-03-24T12:10:01.107293+0000","last_became_active":"2024-03-24T12:09:23.145146+0000","last_became_peered":"2024-03-24T12:09:23.145146+0000","last_unstale":"2024-03-24T12:10:01.107293+0000","last_undegraded":"2024-03-24T12:10:01.107293+0000","last_fullsized":"2024-03-24T12:10:01.107293+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T20:29:27.507482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"68'8","reported_seq":50,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:36.501657+0000","last_change":"2024-03-24T12:09:25.103540+0000","last_active":"2024-03-24T12:10:36.501657+0000","last_peered":"2024-03-24T12:10:36.501657+0000","last_clean":"2024-03-24T12:10:36.501657+0000","last_became_active":"2024-03-24T12:09:25.103180+0000","last_became_peered":"2024-03-24T12:09:25.103180+0000","last_unstale":"2024-03-24T12:10:36.501657+0000","last_undegraded":"2024-03-24T12:10:36.501657+0000","last_fullsized":"2024-03-24T12:10:36.501657+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:04:02.081956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317132+0000","last_change":"2024-03-24T12:09:27.116381+0000","last_active":"2024-03-24T12:10:01.317132+0000","last_peered":"2024-03-24T12:10:01.317132+0000","last_clean":"2024-03-24T12:10:01.317132+0000","last_became_active":"2024-03-24T12:09:27.115964+0000","last_became_peered":"2024-03-24T12:09:27.115964+0000","last_unstale":"2024-03-24T12:10:01.317132+0000","last_undegraded":"2024-03-24T12:10:01.317132+0000","last_fullsized":"2024-03-24T12:10:01.317132+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:16:03.870130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.116590+0000","last_change":"2024-03-24T12:09:20.825958+0000","last_active":"2024-03-24T12:10:01.116590+0000","last_peered":"2024-03-24T12:10:01.116590+0000","last_clean":"2024-03-24T12:10:01.116590+0000","last_became_active":"2024-03-24T12:09:20.825639+0000","last_became_peered":"2024-03-24T12:09:20.825639+0000","last_unstale":"2024-03-24T12:10:01.116590+0000","last_undegraded":"2024-03-24T12:10:01.116590+0000","last_fullsized":"2024-03-24T12:10:01.116590+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T22:09:13.396812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"68'12","reported_seq":70,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997380+0000","last_change":"2024-03-24T12:09:46.968266+0000","last_active":"2024-03-24T12:10:00.997380+0000","last_peered":"2024-03-24T12:10:00.997380+0000","last_clean":"2024-03-24T12:10:00.997380+0000","last_became_active":"2024-03-24T12:09:46.960585+0000","last_became_peered":"2024-03-24T12:09:46.960585+0000","last_unstale":"2024-03-24T12:10:00.997380+0000","last_undegraded":"2024-03-24T12:10:00.997380+0000","last_fullsized":"2024-03-24T12:10:00.997380+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:37:34.452248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.112200+0000","last_change":"2024-03-24T12:09:25.102613+0000","last_active":"2024-03-24T12:10:01.112200+0000","last_peered":"2024-03-24T12:10:01.112200+0000","last_clean":"2024-03-24T12:10:01.112200+0000","last_became_active":"2024-03-24T12:09:25.102224+0000","last_became_peered":"2024-03-24T12:09:25.102224+0000","last_unstale":"2024-03-24T12:10:01.112200+0000","last_undegraded":"2024-03-24T12:10:01.112200+0000","last_fullsized":"2024-03-24T12:10:01.112200+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T23:04:06.272333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.115781+0000","last_change":"2024-03-24T12:09:27.119327+0000","last_active":"2024-03-24T12:10:01.115781+0000","last_peered":"2024-03-24T12:10:01.115781+0000","last_clean":"2024-03-24T12:10:01.115781+0000","last_became_active":"2024-03-24T12:09:27.119062+0000","last_became_peered":"2024-03-24T12:09:27.119062+0000","last_unstale":"2024-03-24T12:10:01.115781+0000","last_undegraded":"2024-03-24T12:10:01.115781+0000","last_fullsized":"2024-03-24T12:10:01.115781+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:54:06.558279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.112699+0000","last_change":"2024-03-24T12:09:25.100273+0000","last_active":"2024-03-24T12:10:01.112699+0000","last_peered":"2024-03-24T12:10:01.112699+0000","last_clean":"2024-03-24T12:10:01.112699+0000","last_became_active":"2024-03-24T12:09:25.099235+0000","last_became_peered":"2024-03-24T12:09:25.099235+0000","last_unstale":"2024-03-24T12:10:01.112699+0000","last_undegraded":"2024-03-24T12:10:01.112699+0000","last_fullsized":"2024-03-24T12:10:01.112699+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:42:13.058472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.116588+0000","last_change":"2024-03-24T12:09:20.829111+0000","last_active":"2024-03-24T12:10:01.116588+0000","last_peered":"2024-03-24T12:10:01.116588+0000","last_clean":"2024-03-24T12:10:01.116588+0000","last_became_active":"2024-03-24T12:09:20.828736+0000","last_became_peered":"2024-03-24T12:09:20.828736+0000","last_unstale":"2024-03-24T12:10:01.116588+0000","last_undegraded":"2024-03-24T12:10:01.116588+0000","last_fullsized":"2024-03-24T12:10:01.116588+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T20:39:57.081827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"68'19","reported_seq":60,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.316828+0000","last_change":"2024-03-24T12:09:23.144173+0000","last_active":"2024-03-24T12:10:01.316828+0000","last_peered":"2024-03-24T12:10:01.316828+0000","last_clean":"2024-03-24T12:10:01.316828+0000","last_became_active":"2024-03-24T12:09:23.143716+0000","last_became_peered":"2024-03-24T12:09:23.143716+0000","last_unstale":"2024-03-24T12:10:01.316828+0000","last_undegraded":"2024-03-24T12:10:01.316828+0000","last_fullsized":"2024-03-24T12:10:01.316828+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:50:54.248390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.993951+0000","last_change":"2024-03-24T12:09:27.112975+0000","last_active":"2024-03-24T12:10:00.993951+0000","last_peered":"2024-03-24T12:10:00.993951+0000","last_clean":"2024-03-24T12:10:00.993951+0000","last_became_active":"2024-03-24T12:09:27.112571+0000","last_became_peered":"2024-03-24T12:09:27.112571+0000","last_unstale":"2024-03-24T12:10:00.993951+0000","last_undegraded":"2024-03-24T12:10:00.993951+0000","last_fullsized":"2024-03-24T12:10:00.993951+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:26:30.958462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.112726+0000","last_change":"2024-03-24T12:09:25.095768+0000","last_active":"2024-03-24T12:10:01.112726+0000","last_peered":"2024-03-24T12:10:01.112726+0000","last_clean":"2024-03-24T12:10:01.112726+0000","last_became_active":"2024-03-24T12:09:25.095444+0000","last_became_peered":"2024-03-24T12:09:25.095444+0000","last_unstale":"2024-03-24T12:10:01.112726+0000","last_undegraded":"2024-03-24T12:10:01.112726+0000","last_fullsized":"2024-03-24T12:10:01.112726+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T20:31:15.096425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107629+0000","last_change":"2024-03-24T12:09:20.819784+0000","last_active":"2024-03-24T12:10:01.107629+0000","last_peered":"2024-03-24T12:10:01.107629+0000","last_clean":"2024-03-24T12:10:01.107629+0000","last_became_active":"2024-03-24T12:09:20.819447+0000","last_became_peered":"2024-03-24T12:09:20.819447+0000","last_unstale":"2024-03-24T12:10:01.107629+0000","last_undegraded":"2024-03-24T12:10:01.107629+0000","last_fullsized":"2024-03-24T12:10:01.107629+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:13:09.930323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.995793+0000","last_change":"2024-03-24T12:09:23.080500+0000","last_active":"2024-03-24T12:10:00.995793+0000","last_peered":"2024-03-24T12:10:00.995793+0000","last_clean":"2024-03-24T12:10:00.995793+0000","last_became_active":"2024-03-24T12:09:23.079806+0000","last_became_peered":"2024-03-24T12:09:23.079806+0000","last_unstale":"2024-03-24T12:10:00.995793+0000","last_undegraded":"2024-03-24T12:10:00.995793+0000","last_fullsized":"2024-03-24T12:10:00.995793+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:27:35.278225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317861+0000","last_change":"2024-03-24T12:09:27.114005+0000","last_active":"2024-03-24T12:10:01.317861+0000","last_peered":"2024-03-24T12:10:01.317861+0000","last_clean":"2024-03-24T12:10:01.317861+0000","last_became_active":"2024-03-24T12:09:27.113656+0000","last_became_peered":"2024-03-24T12:09:27.113656+0000","last_unstale":"2024-03-24T12:10:01.317861+0000","last_undegraded":"2024-03-24T12:10:01.317861+0000","last_fullsized":"2024-03-24T12:10:01.317861+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:30:35.688034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"64'2","reported_seq":44,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107484+0000","last_change":"2024-03-24T12:09:24.090689+0000","last_active":"2024-03-24T12:10:01.107484+0000","last_peered":"2024-03-24T12:10:01.107484+0000","last_clean":"2024-03-24T12:10:01.107484+0000","last_became_active":"2024-03-24T12:09:19.818583+0000","last_became_peered":"2024-03-24T12:09:19.818583+0000","last_unstale":"2024-03-24T12:10:01.107484+0000","last_undegraded":"2024-03-24T12:10:01.107484+0000","last_fullsized":"2024-03-24T12:10:01.107484+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:18.800675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:18.800675+0000","last_clean_scrub_stamp":"2024-03-24T12:09:18.800675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:44:54.337302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022609079999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"61'1","reported_seq":39,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997659+0000","last_change":"2024-03-24T12:09:20.820979+0000","last_active":"2024-03-24T12:10:00.997659+0000","last_peered":"2024-03-24T12:10:00.997659+0000","last_clean":"2024-03-24T12:10:00.997659+0000","last_became_active":"2024-03-24T12:09:20.820649+0000","last_became_peered":"2024-03-24T12:09:20.820649+0000","last_unstale":"2024-03-24T12:10:00.997659+0000","last_undegraded":"2024-03-24T12:10:00.997659+0000","last_fullsized":"2024-03-24T12:10:00.997659+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:16:02.180081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":875,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"68'28","reported_seq":94,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:36.501418+0000","last_change":"2024-03-24T12:09:23.088242+0000","last_active":"2024-03-24T12:10:36.501418+0000","last_peered":"2024-03-24T12:10:36.501418+0000","last_clean":"2024-03-24T12:10:36.501418+0000","last_became_active":"2024-03-24T12:09:23.087959+0000","last_became_peered":"2024-03-24T12:09:23.087959+0000","last_unstale":"2024-03-24T12:10:36.501418+0000","last_undegraded":"2024-03-24T12:10:36.501418+0000","last_fullsized":"2024-03-24T12:10:36.501418+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:26:06.935634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.994408+0000","last_change":"2024-03-24T12:09:25.105808+0000","last_active":"2024-03-24T12:10:00.994408+0000","last_peered":"2024-03-24T12:10:00.994408+0000","last_clean":"2024-03-24T12:10:00.994408+0000","last_became_active":"2024-03-24T12:09:25.105394+0000","last_became_peered":"2024-03-24T12:09:25.105394+0000","last_unstale":"2024-03-24T12:10:00.994408+0000","last_undegraded":"2024-03-24T12:10:00.994408+0000","last_fullsized":"2024-03-24T12:10:00.994408+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T23:39:21.837785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.116678+0000","last_change":"2024-03-24T12:09:27.151051+0000","last_active":"2024-03-24T12:10:01.116678+0000","last_peered":"2024-03-24T12:10:01.116678+0000","last_clean":"2024-03-24T12:10:01.116678+0000","last_became_active":"2024-03-24T12:09:27.150216+0000","last_became_peered":"2024-03-24T12:09:27.150216+0000","last_unstale":"2024-03-24T12:10:01.116678+0000","last_undegraded":"2024-03-24T12:10:01.116678+0000","last_fullsized":"2024-03-24T12:10:01.116678+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:24:33.029899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.316866+0000","last_change":"2024-03-24T12:09:21.445065+0000","last_active":"2024-03-24T12:10:01.316866+0000","last_peered":"2024-03-24T12:10:01.316866+0000","last_clean":"2024-03-24T12:10:01.316866+0000","last_became_active":"2024-03-24T12:09:21.444217+0000","last_became_peered":"2024-03-24T12:09:21.444217+0000","last_unstale":"2024-03-24T12:10:01.316866+0000","last_undegraded":"2024-03-24T12:10:01.316866+0000","last_fullsized":"2024-03-24T12:10:01.316866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:21:10.451340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.112433+0000","last_change":"2024-03-24T12:09:24.084016+0000","last_active":"2024-03-24T12:10:01.112433+0000","last_peered":"2024-03-24T12:10:01.112433+0000","last_clean":"2024-03-24T12:10:01.112433+0000","last_became_active":"2024-03-24T12:09:19.816551+0000","last_became_peered":"2024-03-24T12:09:19.816551+0000","last_unstale":"2024-03-24T12:10:01.112433+0000","last_undegraded":"2024-03-24T12:10:01.112433+0000","last_fullsized":"2024-03-24T12:10:01.112433+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:18.800675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:18.800675+0000","last_clean_scrub_stamp":"2024-03-24T12:09:18.800675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:00:19.277567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065894900000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"68'13","reported_seq":56,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.318235+0000","last_change":"2024-03-24T12:09:23.081979+0000","last_active":"2024-03-24T12:10:01.318235+0000","last_peered":"2024-03-24T12:10:01.318235+0000","last_clean":"2024-03-24T12:10:01.318235+0000","last_became_active":"2024-03-24T12:09:23.080958+0000","last_became_peered":"2024-03-24T12:09:23.080958+0000","last_unstale":"2024-03-24T12:10:01.318235+0000","last_undegraded":"2024-03-24T12:10:01.318235+0000","last_fullsized":"2024-03-24T12:10:01.318235+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:07:11.319443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.112790+0000","last_change":"2024-03-24T12:09:25.100039+0000","last_active":"2024-03-24T12:10:01.112790+0000","last_peered":"2024-03-24T12:10:01.112790+0000","last_clean":"2024-03-24T12:10:01.112790+0000","last_became_active":"2024-03-24T12:09:25.098643+0000","last_became_peered":"2024-03-24T12:09:25.098643+0000","last_unstale":"2024-03-24T12:10:01.112790+0000","last_undegraded":"2024-03-24T12:10:01.112790+0000","last_fullsized":"2024-03-24T12:10:01.112790+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:40:03.778416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107820+0000","last_change":"2024-03-24T12:09:27.118123+0000","last_active":"2024-03-24T12:10:01.107820+0000","last_peered":"2024-03-24T12:10:01.107820+0000","last_clean":"2024-03-24T12:10:01.107820+0000","last_became_active":"2024-03-24T12:09:27.117788+0000","last_became_peered":"2024-03-24T12:09:27.117788+0000","last_unstale":"2024-03-24T12:10:01.107820+0000","last_undegraded":"2024-03-24T12:10:01.107820+0000","last_fullsized":"2024-03-24T12:10:01.107820+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:23:24.436351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":278,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:03.098710+0000","last_change":"2024-03-24T12:09:10.475623+0000","last_active":"2024-03-24T12:10:03.098710+0000","last_peered":"2024-03-24T12:10:03.098710+0000","last_clean":"2024-03-24T12:10:03.098710+0000","last_became_active":"2024-03-24T12:09:10.475313+0000","last_became_peered":"2024-03-24T12:09:10.475313+0000","last_unstale":"2024-03-24T12:10:03.098710+0000","last_undegraded":"2024-03-24T12:10:03.098710+0000","last_fullsized":"2024-03-24T12:10:03.098710+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:07:02.602210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:07:02.602210+0000","last_clean_scrub_stamp":"2024-03-24T12:07:02.602210+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:53:43.283275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317252+0000","last_change":"2024-03-24T12:09:20.821957+0000","last_active":"2024-03-24T12:10:01.317252+0000","last_peered":"2024-03-24T12:10:01.317252+0000","last_clean":"2024-03-24T12:10:01.317252+0000","last_became_active":"2024-03-24T12:09:20.821651+0000","last_became_peered":"2024-03-24T12:09:20.821651+0000","last_unstale":"2024-03-24T12:10:01.317252+0000","last_undegraded":"2024-03-24T12:10:01.317252+0000","last_fullsized":"2024-03-24T12:10:01.317252+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:20:03.531446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"68'16","reported_seq":68,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:36.501662+0000","last_change":"2024-03-24T12:09:23.145396+0000","last_active":"2024-03-24T12:10:36.501662+0000","last_peered":"2024-03-24T12:10:36.501662+0000","last_clean":"2024-03-24T12:10:36.501662+0000","last_became_active":"2024-03-24T12:09:23.144993+0000","last_became_peered":"2024-03-24T12:09:23.144993+0000","last_unstale":"2024-03-24T12:10:36.501662+0000","last_undegraded":"2024-03-24T12:10:36.501662+0000","last_fullsized":"2024-03-24T12:10:36.501662+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:22:14.043318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.116407+0000","last_change":"2024-03-24T12:09:25.104997+0000","last_active":"2024-03-24T12:10:01.116407+0000","last_peered":"2024-03-24T12:10:01.116407+0000","last_clean":"2024-03-24T12:10:01.116407+0000","last_became_active":"2024-03-24T12:09:25.104683+0000","last_became_peered":"2024-03-24T12:09:25.104683+0000","last_unstale":"2024-03-24T12:10:01.116407+0000","last_undegraded":"2024-03-24T12:10:01.116407+0000","last_fullsized":"2024-03-24T12:10:01.116407+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:31:23.470732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317877+0000","last_change":"2024-03-24T12:09:27.117597+0000","last_active":"2024-03-24T12:10:01.317877+0000","last_peered":"2024-03-24T12:10:01.317877+0000","last_clean":"2024-03-24T12:10:01.317877+0000","last_became_active":"2024-03-24T12:09:27.117243+0000","last_became_peered":"2024-03-24T12:09:27.117243+0000","last_unstale":"2024-03-24T12:10:01.317877+0000","last_undegraded":"2024-03-24T12:10:01.317877+0000","last_fullsized":"2024-03-24T12:10:01.317877+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:18:16.692691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.994134+0000","last_change":"2024-03-24T12:09:20.822275+0000","last_active":"2024-03-24T12:10:00.994134+0000","last_peered":"2024-03-24T12:10:00.994134+0000","last_clean":"2024-03-24T12:10:00.994134+0000","last_became_active":"2024-03-24T12:09:20.821920+0000","last_became_peered":"2024-03-24T12:09:20.821920+0000","last_unstale":"2024-03-24T12:10:00.994134+0000","last_undegraded":"2024-03-24T12:10:00.994134+0000","last_fullsized":"2024-03-24T12:10:00.994134+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:52:20.544068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"69'5","reported_seq":119,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:40.942374+0000","last_change":"2024-03-24T12:09:24.089836+0000","last_active":"2024-03-24T12:10:40.942374+0000","last_peered":"2024-03-24T12:10:40.942374+0000","last_clean":"2024-03-24T12:10:40.942374+0000","last_became_active":"2024-03-24T12:09:19.818613+0000","last_became_peered":"2024-03-24T12:09:19.818613+0000","last_unstale":"2024-03-24T12:10:40.942374+0000","last_undegraded":"2024-03-24T12:10:40.942374+0000","last_fullsized":"2024-03-24T12:10:40.942374+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:18.800675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:18.800675+0000","last_clean_scrub_stamp":"2024-03-24T12:09:18.800675+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:32:33.066054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042976100000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":74,"num_read_kb":69,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"68'12","reported_seq":47,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.994676+0000","last_change":"2024-03-24T12:09:23.079972+0000","last_active":"2024-03-24T12:10:00.994676+0000","last_peered":"2024-03-24T12:10:00.994676+0000","last_clean":"2024-03-24T12:10:00.994676+0000","last_became_active":"2024-03-24T12:09:23.079579+0000","last_became_peered":"2024-03-24T12:09:23.079579+0000","last_unstale":"2024-03-24T12:10:00.994676+0000","last_undegraded":"2024-03-24T12:10:00.994676+0000","last_fullsized":"2024-03-24T12:10:00.994676+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-26T00:01:44.813315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.108158+0000","last_change":"2024-03-24T12:09:25.106326+0000","last_active":"2024-03-24T12:10:01.108158+0000","last_peered":"2024-03-24T12:10:01.108158+0000","last_clean":"2024-03-24T12:10:01.108158+0000","last_became_active":"2024-03-24T12:09:25.106033+0000","last_became_peered":"2024-03-24T12:09:25.106033+0000","last_unstale":"2024-03-24T12:10:01.108158+0000","last_undegraded":"2024-03-24T12:10:01.108158+0000","last_fullsized":"2024-03-24T12:10:01.108158+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:10:38.677188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.316932+0000","last_change":"2024-03-24T12:09:27.149981+0000","last_active":"2024-03-24T12:10:01.316932+0000","last_peered":"2024-03-24T12:10:01.316932+0000","last_clean":"2024-03-24T12:10:01.316932+0000","last_became_active":"2024-03-24T12:09:27.149577+0000","last_became_peered":"2024-03-24T12:09:27.149577+0000","last_unstale":"2024-03-24T12:10:01.316932+0000","last_undegraded":"2024-03-24T12:10:01.316932+0000","last_fullsized":"2024-03-24T12:10:01.316932+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T22:29:28.129633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.318210+0000","last_change":"2024-03-24T12:09:21.444784+0000","last_active":"2024-03-24T12:10:01.318210+0000","last_peered":"2024-03-24T12:10:01.318210+0000","last_clean":"2024-03-24T12:10:01.318210+0000","last_became_active":"2024-03-24T12:09:21.444227+0000","last_became_peered":"2024-03-24T12:09:21.444227+0000","last_unstale":"2024-03-24T12:10:01.318210+0000","last_undegraded":"2024-03-24T12:10:01.318210+0000","last_fullsized":"2024-03-24T12:10:01.318210+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:17:17.206900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"68'19","reported_seq":65,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.995386+0000","last_change":"2024-03-24T12:09:23.144276+0000","last_active":"2024-03-24T12:10:00.995386+0000","last_peered":"2024-03-24T12:10:00.995386+0000","last_clean":"2024-03-24T12:10:00.995386+0000","last_became_active":"2024-03-24T12:09:23.143963+0000","last_became_peered":"2024-03-24T12:09:23.143963+0000","last_unstale":"2024-03-24T12:10:00.995386+0000","last_undegraded":"2024-03-24T12:10:00.995386+0000","last_fullsized":"2024-03-24T12:10:00.995386+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:17:08.898988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.316852+0000","last_change":"2024-03-24T12:09:25.097995+0000","last_active":"2024-03-24T12:10:01.316852+0000","last_peered":"2024-03-24T12:10:01.316852+0000","last_clean":"2024-03-24T12:10:01.316852+0000","last_became_active":"2024-03-24T12:09:25.097693+0000","last_became_peered":"2024-03-24T12:09:25.097693+0000","last_unstale":"2024-03-24T12:10:01.316852+0000","last_undegraded":"2024-03-24T12:10:01.316852+0000","last_fullsized":"2024-03-24T12:10:01.316852+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:31:42.256234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.993874+0000","last_change":"2024-03-24T12:09:27.119367+0000","last_active":"2024-03-24T12:10:00.993874+0000","last_peered":"2024-03-24T12:10:00.993874+0000","last_clean":"2024-03-24T12:10:00.993874+0000","last_became_active":"2024-03-24T12:09:27.119064+0000","last_became_peered":"2024-03-24T12:09:27.119064+0000","last_unstale":"2024-03-24T12:10:00.993874+0000","last_undegraded":"2024-03-24T12:10:00.993874+0000","last_fullsized":"2024-03-24T12:10:00.993874+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:41:48.578066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107853+0000","last_change":"2024-03-24T12:09:21.444520+0000","last_active":"2024-03-24T12:10:01.107853+0000","last_peered":"2024-03-24T12:10:01.107853+0000","last_clean":"2024-03-24T12:10:01.107853+0000","last_became_active":"2024-03-24T12:09:21.444158+0000","last_became_peered":"2024-03-24T12:09:21.444158+0000","last_unstale":"2024-03-24T12:10:01.107853+0000","last_undegraded":"2024-03-24T12:10:01.107853+0000","last_fullsized":"2024-03-24T12:10:01.107853+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T15:57:35.914727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"68'10","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.112399+0000","last_change":"2024-03-24T12:09:46.969753+0000","last_active":"2024-03-24T12:10:01.112399+0000","last_peered":"2024-03-24T12:10:01.112399+0000","last_clean":"2024-03-24T12:10:01.112399+0000","last_became_active":"2024-03-24T12:09:46.961937+0000","last_became_peered":"2024-03-24T12:09:46.961937+0000","last_unstale":"2024-03-24T12:10:01.112399+0000","last_undegraded":"2024-03-24T12:10:01.112399+0000","last_fullsized":"2024-03-24T12:10:01.112399+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T17:16:34.746853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"68'8","reported_seq":50,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:36.501418+0000","last_change":"2024-03-24T12:09:25.102930+0000","last_active":"2024-03-24T12:10:36.501418+0000","last_peered":"2024-03-24T12:10:36.501418+0000","last_clean":"2024-03-24T12:10:36.501418+0000","last_became_active":"2024-03-24T12:09:25.102575+0000","last_became_peered":"2024-03-24T12:09:25.102575+0000","last_unstale":"2024-03-24T12:10:36.501418+0000","last_undegraded":"2024-03-24T12:10:36.501418+0000","last_fullsized":"2024-03-24T12:10:36.501418+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:36:26.190463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.116271+0000","last_change":"2024-03-24T12:09:27.150769+0000","last_active":"2024-03-24T12:10:01.116271+0000","last_peered":"2024-03-24T12:10:01.116271+0000","last_clean":"2024-03-24T12:10:01.116271+0000","last_became_active":"2024-03-24T12:09:27.150362+0000","last_became_peered":"2024-03-24T12:09:27.150362+0000","last_unstale":"2024-03-24T12:10:01.116271+0000","last_undegraded":"2024-03-24T12:10:01.116271+0000","last_fullsized":"2024-03-24T12:10:01.116271+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:57:32.489010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"61'1","reported_seq":34,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.994810+0000","last_change":"2024-03-24T12:09:20.820873+0000","last_active":"2024-03-24T12:10:00.994810+0000","last_peered":"2024-03-24T12:10:00.994810+0000","last_clean":"2024-03-24T12:10:00.994810+0000","last_became_active":"2024-03-24T12:09:20.819577+0000","last_became_peered":"2024-03-24T12:09:20.819577+0000","last_unstale":"2024-03-24T12:10:00.994810+0000","last_undegraded":"2024-03-24T12:10:00.994810+0000","last_fullsized":"2024-03-24T12:10:00.994810+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:42:49.044131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"68'14","reported_seq":50,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997256+0000","last_change":"2024-03-24T12:09:23.077103+0000","last_active":"2024-03-24T12:10:00.997256+0000","last_peered":"2024-03-24T12:10:00.997256+0000","last_clean":"2024-03-24T12:10:00.997256+0000","last_became_active":"2024-03-24T12:09:23.076572+0000","last_became_peered":"2024-03-24T12:09:23.076572+0000","last_unstale":"2024-03-24T12:10:00.997256+0000","last_undegraded":"2024-03-24T12:10:00.997256+0000","last_fullsized":"2024-03-24T12:10:00.997256+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T15:08:15.084609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"68'8","reported_seq":50,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:36.501473+0000","last_change":"2024-03-24T12:09:25.102449+0000","last_active":"2024-03-24T12:10:36.501473+0000","last_peered":"2024-03-24T12:10:36.501473+0000","last_clean":"2024-03-24T12:10:36.501473+0000","last_became_active":"2024-03-24T12:09:25.102099+0000","last_became_peered":"2024-03-24T12:09:25.102099+0000","last_unstale":"2024-03-24T12:10:36.501473+0000","last_undegraded":"2024-03-24T12:10:36.501473+0000","last_fullsized":"2024-03-24T12:10:36.501473+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T15:21:24.119081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997142+0000","last_change":"2024-03-24T12:09:27.116459+0000","last_active":"2024-03-24T12:10:00.997142+0000","last_peered":"2024-03-24T12:10:00.997142+0000","last_clean":"2024-03-24T12:10:00.997142+0000","last_became_active":"2024-03-24T12:09:27.116175+0000","last_became_peered":"2024-03-24T12:09:27.116175+0000","last_unstale":"2024-03-24T12:10:00.997142+0000","last_undegraded":"2024-03-24T12:10:00.997142+0000","last_fullsized":"2024-03-24T12:10:00.997142+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T23:10:56.728970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.318141+0000","last_change":"2024-03-24T12:09:20.828927+0000","last_active":"2024-03-24T12:10:01.318141+0000","last_peered":"2024-03-24T12:10:01.318141+0000","last_clean":"2024-03-24T12:10:01.318141+0000","last_became_active":"2024-03-24T12:09:20.828616+0000","last_became_peered":"2024-03-24T12:09:20.828616+0000","last_unstale":"2024-03-24T12:10:01.318141+0000","last_undegraded":"2024-03-24T12:10:01.318141+0000","last_fullsized":"2024-03-24T12:10:01.318141+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:14:54.225013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"68'18","reported_seq":61,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317725+0000","last_change":"2024-03-24T12:09:23.145425+0000","last_active":"2024-03-24T12:10:01.317725+0000","last_peered":"2024-03-24T12:10:01.317725+0000","last_clean":"2024-03-24T12:10:01.317725+0000","last_became_active":"2024-03-24T12:09:23.145076+0000","last_became_peered":"2024-03-24T12:09:23.145076+0000","last_unstale":"2024-03-24T12:10:01.317725+0000","last_undegraded":"2024-03-24T12:10:01.317725+0000","last_fullsized":"2024-03-24T12:10:01.317725+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T15:14:42.999283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997048+0000","last_change":"2024-03-24T12:09:25.098716+0000","last_active":"2024-03-24T12:10:00.997048+0000","last_peered":"2024-03-24T12:10:00.997048+0000","last_clean":"2024-03-24T12:10:00.997048+0000","last_became_active":"2024-03-24T12:09:25.098286+0000","last_became_peered":"2024-03-24T12:09:25.098286+0000","last_unstale":"2024-03-24T12:10:00.997048+0000","last_undegraded":"2024-03-24T12:10:00.997048+0000","last_fullsized":"2024-03-24T12:10:00.997048+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T13:15:47.356055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.107605+0000","last_change":"2024-03-24T12:09:27.107290+0000","last_active":"2024-03-24T12:10:01.107605+0000","last_peered":"2024-03-24T12:10:01.107605+0000","last_clean":"2024-03-24T12:10:01.107605+0000","last_became_active":"2024-03-24T12:09:27.106990+0000","last_became_peered":"2024-03-24T12:09:27.106990+0000","last_unstale":"2024-03-24T12:10:01.107605+0000","last_undegraded":"2024-03-24T12:10:01.107605+0000","last_fullsized":"2024-03-24T12:10:01.107605+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T23:05:22.631547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"68'8","reported_seq":50,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:36.501646+0000","last_change":"2024-03-24T12:09:25.098147+0000","last_active":"2024-03-24T12:10:36.501646+0000","last_peered":"2024-03-24T12:10:36.501646+0000","last_clean":"2024-03-24T12:10:36.501646+0000","last_became_active":"2024-03-24T12:09:25.097418+0000","last_became_peered":"2024-03-24T12:09:25.097418+0000","last_unstale":"2024-03-24T12:10:36.501646+0000","last_undegraded":"2024-03-24T12:10:36.501646+0000","last_fullsized":"2024-03-24T12:10:36.501646+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:14:17.359686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317621+0000","last_change":"2024-03-24T12:09:20.827094+0000","last_active":"2024-03-24T12:10:01.317621+0000","last_peered":"2024-03-24T12:10:01.317621+0000","last_clean":"2024-03-24T12:10:01.317621+0000","last_became_active":"2024-03-24T12:09:20.824753+0000","last_became_peered":"2024-03-24T12:09:20.824753+0000","last_unstale":"2024-03-24T12:10:01.317621+0000","last_undegraded":"2024-03-24T12:10:01.317621+0000","last_fullsized":"2024-03-24T12:10:01.317621+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T22:55:10.534182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"68'15","reported_seq":54,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.318418+0000","last_change":"2024-03-24T12:09:23.088102+0000","last_active":"2024-03-24T12:10:01.318418+0000","last_peered":"2024-03-24T12:10:01.318418+0000","last_clean":"2024-03-24T12:10:01.318418+0000","last_became_active":"2024-03-24T12:09:23.087739+0000","last_became_peered":"2024-03-24T12:09:23.087739+0000","last_unstale":"2024-03-24T12:10:01.318418+0000","last_undegraded":"2024-03-24T12:10:01.318418+0000","last_fullsized":"2024-03-24T12:10:01.318418+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T12:25:35.825369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317574+0000","last_change":"2024-03-24T12:09:27.148982+0000","last_active":"2024-03-24T12:10:01.317574+0000","last_peered":"2024-03-24T12:10:01.317574+0000","last_clean":"2024-03-24T12:10:01.317574+0000","last_became_active":"2024-03-24T12:09:27.148641+0000","last_became_peered":"2024-03-24T12:09:27.148641+0000","last_unstale":"2024-03-24T12:10:01.317574+0000","last_undegraded":"2024-03-24T12:10:01.317574+0000","last_fullsized":"2024-03-24T12:10:01.317574+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:25:21.697003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.108127+0000","last_change":"2024-03-24T12:09:25.099260+0000","last_active":"2024-03-24T12:10:01.108127+0000","last_peered":"2024-03-24T12:10:01.108127+0000","last_clean":"2024-03-24T12:10:01.108127+0000","last_became_active":"2024-03-24T12:09:25.098964+0000","last_became_peered":"2024-03-24T12:09:25.098964+0000","last_unstale":"2024-03-24T12:10:01.108127+0000","last_undegraded":"2024-03-24T12:10:01.108127+0000","last_fullsized":"2024-03-24T12:10:01.108127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T20:46:40.805707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.997045+0000","last_change":"2024-03-24T12:09:21.445910+0000","last_active":"2024-03-24T12:10:00.997045+0000","last_peered":"2024-03-24T12:10:00.997045+0000","last_clean":"2024-03-24T12:10:00.997045+0000","last_became_active":"2024-03-24T12:09:21.445619+0000","last_became_peered":"2024-03-24T12:09:21.445619+0000","last_unstale":"2024-03-24T12:10:00.997045+0000","last_undegraded":"2024-03-24T12:10:00.997045+0000","last_fullsized":"2024-03-24T12:10:00.997045+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:54:32.662859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"68'11","reported_seq":48,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.999156+0000","last_change":"2024-03-24T12:09:23.077392+0000","last_active":"2024-03-24T12:10:00.999156+0000","last_peered":"2024-03-24T12:10:00.999156+0000","last_clean":"2024-03-24T12:10:00.999156+0000","last_became_active":"2024-03-24T12:09:23.076962+0000","last_became_peered":"2024-03-24T12:09:23.076962+0000","last_unstale":"2024-03-24T12:10:00.999156+0000","last_undegraded":"2024-03-24T12:10:00.999156+0000","last_fullsized":"2024-03-24T12:10:00.999156+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:15:37.747799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.113290+0000","last_change":"2024-03-24T12:09:27.116400+0000","last_active":"2024-03-24T12:10:01.113290+0000","last_peered":"2024-03-24T12:10:01.113290+0000","last_clean":"2024-03-24T12:10:01.113290+0000","last_became_active":"2024-03-24T12:09:27.116084+0000","last_became_peered":"2024-03-24T12:09:27.116084+0000","last_unstale":"2024-03-24T12:10:01.113290+0000","last_undegraded":"2024-03-24T12:10:01.113290+0000","last_fullsized":"2024-03-24T12:10:01.113290+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T15:36:58.981364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.318430+0000","last_change":"2024-03-24T12:09:25.099499+0000","last_active":"2024-03-24T12:10:01.318430+0000","last_peered":"2024-03-24T12:10:01.318430+0000","last_clean":"2024-03-24T12:10:01.318430+0000","last_became_active":"2024-03-24T12:09:25.099175+0000","last_became_peered":"2024-03-24T12:09:25.099175+0000","last_unstale":"2024-03-24T12:10:01.318430+0000","last_undegraded":"2024-03-24T12:10:01.318430+0000","last_fullsized":"2024-03-24T12:10:01.318430+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T23:26:33.849514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.316958+0000","last_change":"2024-03-24T12:09:20.818832+0000","last_active":"2024-03-24T12:10:01.316958+0000","last_peered":"2024-03-24T12:10:01.316958+0000","last_clean":"2024-03-24T12:10:01.316958+0000","last_became_active":"2024-03-24T12:09:20.818530+0000","last_became_peered":"2024-03-24T12:09:20.818530+0000","last_unstale":"2024-03-24T12:10:01.316958+0000","last_undegraded":"2024-03-24T12:10:01.316958+0000","last_fullsized":"2024-03-24T12:10:01.316958+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T18:46:08.663561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"68'17","reported_seq":57,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.999665+0000","last_change":"2024-03-24T12:09:23.082282+0000","last_active":"2024-03-24T12:10:00.999665+0000","last_peered":"2024-03-24T12:10:00.999665+0000","last_clean":"2024-03-24T12:10:00.999665+0000","last_became_active":"2024-03-24T12:09:23.081949+0000","last_became_peered":"2024-03-24T12:09:23.081949+0000","last_unstale":"2024-03-24T12:10:00.999665+0000","last_undegraded":"2024-03-24T12:10:00.999665+0000","last_fullsized":"2024-03-24T12:10:00.999665+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T16:21:38.466360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.999505+0000","last_change":"2024-03-24T12:09:27.111186+0000","last_active":"2024-03-24T12:10:00.999505+0000","last_peered":"2024-03-24T12:10:00.999505+0000","last_clean":"2024-03-24T12:10:00.999505+0000","last_became_active":"2024-03-24T12:09:27.110879+0000","last_became_peered":"2024-03-24T12:09:27.110879+0000","last_unstale":"2024-03-24T12:10:00.999505+0000","last_undegraded":"2024-03-24T12:10:00.999505+0000","last_fullsized":"2024-03-24T12:10:00.999505+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:26.090685+0000","last_clean_scrub_stamp":"2024-03-24T12:09:26.090685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T23:38:00.319682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"68'8","reported_seq":50,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:36.501414+0000","last_change":"2024-03-24T12:09:25.100778+0000","last_active":"2024-03-24T12:10:36.501414+0000","last_peered":"2024-03-24T12:10:36.501414+0000","last_clean":"2024-03-24T12:10:36.501414+0000","last_became_active":"2024-03-24T12:09:25.100474+0000","last_became_peered":"2024-03-24T12:09:25.100474+0000","last_unstale":"2024-03-24T12:10:36.501414+0000","last_undegraded":"2024-03-24T12:10:36.501414+0000","last_fullsized":"2024-03-24T12:10:36.501414+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:24.076089+0000","last_clean_scrub_stamp":"2024-03-24T12:09:24.076089+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T14:34:39.236366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"61'1","reported_seq":39,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:01.317372+0000","last_change":"2024-03-24T12:09:20.820784+0000","last_active":"2024-03-24T12:10:01.317372+0000","last_peered":"2024-03-24T12:10:01.317372+0000","last_clean":"2024-03-24T12:10:01.317372+0000","last_became_active":"2024-03-24T12:09:20.820492+0000","last_became_peered":"2024-03-24T12:09:20.820492+0000","last_unstale":"2024-03-24T12:10:01.317372+0000","last_undegraded":"2024-03-24T12:10:01.317372+0000","last_fullsized":"2024-03-24T12:10:01.317372+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:19.807295+0000","last_clean_scrub_stamp":"2024-03-24T12:09:19.807295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T21:30:28.025900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"68'10","reported_seq":44,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-03-24T12:10:00.998137+0000","last_change":"2024-03-24T12:09:23.084398+0000","last_active":"2024-03-24T12:10:00.998137+0000","last_peered":"2024-03-24T12:10:00.998137+0000","last_clean":"2024-03-24T12:10:00.998137+0000","last_became_active":"2024-03-24T12:09:23.083831+0000","last_became_peered":"2024-03-24T12:09:23.083831+0000","last_unstale":"2024-03-24T12:10:00.998137+0000","last_undegraded":"2024-03-24T12:10:00.998137+0000","last_fullsized":"2024-03-24T12:10:00.998137+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-24T12:09:22.057486+0000","last_clean_scrub_stamp":"2024-03-24T12:09:22.057486+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T19:01:00.297743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":383,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1457,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":69,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234005,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28388,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716764,"statfs":{"total":95995035648,"available":95965966336,"internally_reserved":0,"allocated":1404928,"data_stored":928991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495642,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32532,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712620,"statfs":{"total":95995035648,"available":95961722880,"internally_reserved":0,"allocated":1388544,"data_stored":928814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":839680,"data_stored":339013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28396,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716756,"statfs":{"total":95995035648,"available":95965958144,"internally_reserved":0,"allocated":1413120,"data_stored":929993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31956,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713196,"statfs":{"total":95995035648,"available":95962312704,"internally_reserved":0,"allocated":794624,"data_stored":338003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182442,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":831488,"data_stored":338860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018916,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":823296,"data_stored":339436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757279,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31952,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713200,"statfs":{"total":95995035648,"available":95962316800,"internally_reserved":0,"allocated":790528,"data_stored":337884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-24T12:10:45.904 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-24T12:10:45.904 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-24T12:10:45.904 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-24T12:10:45.904 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 shell --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 -- ceph health --format=json 2024-03-24T12:10:46.298 INFO:teuthology.orchestra.run.smithi006.stderr:Inferring config /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/mon.a/config 2024-03-24T12:10:46.422 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:10:46 smithi006 podman[79357]: 2024-03-24 12:10:46.146810298 +0000 UTC m=+1.302685878 container init bba13843ec61747d522d369757ecee450b507e8be0fd8406d985ee0522ede4ae (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T12:10:46.422 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:10:46 smithi006 podman[79357]: 2024-03-24 12:10:46.153313122 +0000 UTC m=+1.309188695 container start bba13843ec61747d522d369757ecee450b507e8be0fd8406d985ee0522ede4ae (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T12:10:46.422 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a[79411]: ts=2024-03-24T12:10:46.166Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-03-24T12:10:46.422 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a[79411]: ts=2024-03-24T12:10:46.166Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-03-24T12:10:46.422 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a[79411]: ts=2024-03-24T12:10:46.167Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.6 port=9094 2024-03-24T12:10:46.422 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a[79411]: ts=2024-03-24T12:10:46.168Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-03-24T12:10:46.422 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a[79411]: ts=2024-03-24T12:10:46.192Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-24T12:10:46.422 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a[79411]: ts=2024-03-24T12:10:46.192Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-24T12:10:46.422 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a[79411]: ts=2024-03-24T12:10:46.195Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-03-24T12:10:46.423 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a[79411]: ts=2024-03-24T12:10:46.195Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-03-24T12:10:46.423 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:10:46 smithi006 bash[79357]: bba13843ec61747d522d369757ecee450b507e8be0fd8406d985ee0522ede4ae 2024-03-24T12:10:46.423 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:10:46 smithi006 systemd[1]: Started Ceph alertmanager.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:10:46.691 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:10:46] ENGINE Bus STOPPING 2024-03-24T12:10:46.691 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:10:46] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-24T12:10:46.691 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:10:46] ENGINE Bus STOPPED 2024-03-24T12:10:46.691 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:10:46] ENGINE Bus STARTING 2024-03-24T12:10:46.691 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:10:46] ENGINE Serving on http://:::9283 2024-03-24T12:10:46.692 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:10:46] ENGINE Bus STARTED 2024-03-24T12:10:46.692 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-mon[30108]: from='client.24608 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T12:10:46.692 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-mon[30108]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:46.692 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:10:46.692 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:46.692 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:46.692 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:46.692 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:46.692 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:46.692 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:46.692 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-24T12:10:46.692 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:46.693 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-mon[41734]: from='client.24608 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T12:10:46.693 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-mon[41734]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:46.693 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:10:46.693 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:46.693 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:46.693 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:46.693 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:46.693 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:46.693 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:46.693 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-24T12:10:46.693 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:46 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:46.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:46 smithi090 ceph-mon[33646]: from='client.24608 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-24T12:10:46.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:46 smithi090 ceph-mon[33646]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:46.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:46 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:10:46.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:46 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:46.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:46 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:46.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:46 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:46.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:46 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:46.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:46 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:46.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:46 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:46.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:46 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-24T12:10:46.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:46 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:47.854 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:47 smithi006 ceph-mon[30108]: Regenerating cephadm self-signed grafana TLS certificates 2024-03-24T12:10:47.854 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:47 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-24T12:10:47.854 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:47 smithi006 ceph-mon[30108]: Deploying daemon grafana.a on smithi090 2024-03-24T12:10:47.854 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:47 smithi006 ceph-mon[41734]: Regenerating cephadm self-signed grafana TLS certificates 2024-03-24T12:10:47.854 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:47 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-24T12:10:47.854 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:47 smithi006 ceph-mon[41734]: Deploying daemon grafana.a on smithi090 2024-03-24T12:10:47.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:47 smithi090 ceph-mon[33646]: Regenerating cephadm self-signed grafana TLS certificates 2024-03-24T12:10:47.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:47 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-24T12:10:47.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:47 smithi090 ceph-mon[33646]: Deploying daemon grafana.a on smithi090 2024-03-24T12:10:48.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.108+0000 7fbca48db700 1 -- 172.21.15.6:0/1762059112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbca0100dc0 msgr2=0x7fbca01011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:48.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.108+0000 7fbca48db700 1 --2- 172.21.15.6:0/1762059112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbca0100dc0 0x7fbca01011a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009990 tx=0x7fbc90009ca0 comp rx=0 tx=0).stop 2024-03-24T12:10:48.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.108+0000 7fbca48db700 1 -- 172.21.15.6:0/1762059112 shutdown_connections 2024-03-24T12:10:48.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.108+0000 7fbca48db700 1 --2- 172.21.15.6:0/1762059112 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbca0105e80 0x7fbca010da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:48.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.108+0000 7fbca48db700 1 --2- 172.21.15.6:0/1762059112 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbca0100dc0 0x7fbca01011a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:48.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.108+0000 7fbca48db700 1 --2- 172.21.15.6:0/1762059112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbca0101770 0x7fbca01057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:48.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.108+0000 7fbca48db700 1 -- 172.21.15.6:0/1762059112 >> 172.21.15.6:0/1762059112 conn(0x7fbca0077ae0 msgr2=0x7fbca0077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:48.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.108+0000 7fbca48db700 1 -- 172.21.15.6:0/1762059112 shutdown_connections 2024-03-24T12:10:48.108 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.108+0000 7fbca48db700 1 -- 172.21.15.6:0/1762059112 wait complete. 2024-03-24T12:10:48.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.109+0000 7fbca48db700 1 Processor -- start 2024-03-24T12:10:48.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.109+0000 7fbca48db700 1 -- start start 2024-03-24T12:10:48.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.109+0000 7fbca48db700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbca0100dc0 0x7fbca01965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:48.110 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.109+0000 7fbca48db700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbca0101770 0x7fbca0196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:48.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.109+0000 7fbca48db700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbca0105e80 0x7fbca019aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:48.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.109+0000 7fbca48db700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca019b4c0 con 0x7fbca0101770 2024-03-24T12:10:48.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.109+0000 7fbca48db700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca019b630 con 0x7fbca0105e80 2024-03-24T12:10:48.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.109+0000 7fbca48db700 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca019b7f0 con 0x7fbca0100dc0 2024-03-24T12:10:48.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.109+0000 7fbc97fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbca0101770 0x7fbca0196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:48.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.109+0000 7fbc97fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbca0101770 0x7fbca0196b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:38034/0 (socket says 172.21.15.6:38034) 2024-03-24T12:10:48.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.110+0000 7fbc9ed9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbca0105e80 0x7fbca019aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:48.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.110+0000 7fbc9e59c700 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbca0100dc0 0x7fbca01965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:48.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.110+0000 7fbc9ed9d700 1 --2- 172.21.15.6:0/3255203402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbca0105e80 0x7fbca019aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.6:50812/0 (socket says 172.21.15.6:50812) 2024-03-24T12:10:48.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.110+0000 7fbc97fff700 1 -- 172.21.15.6:0/3255203402 learned_addr learned my addr 172.21.15.6:0/3255203402 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:10:48.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.110+0000 7fbc9e59c700 1 -- 172.21.15.6:0/3255203402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbca0105e80 msgr2=0x7fbca019aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:48.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.110+0000 7fbc9e59c700 1 --2- 172.21.15.6:0/3255203402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbca0105e80 0x7fbca019aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:48.111 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.110+0000 7fbc9e59c700 1 -- 172.21.15.6:0/3255203402 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbca0101770 msgr2=0x7fbca0196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:48.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.110+0000 7fbc9e59c700 1 --2- 172.21.15.6:0/3255203402 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbca0101770 0x7fbca0196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:48.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.110+0000 7fbc9e59c700 1 -- 172.21.15.6:0/3255203402 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc900095f0 con 0x7fbca0100dc0 2024-03-24T12:10:48.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.110+0000 7fbc9e59c700 1 --2- 172.21.15.6:0/3255203402 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbca0100dc0 0x7fbca01965e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbc900098e0 tx=0x7fbc90005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:48.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.110+0000 7fbc977fe700 1 -- 172.21.15.6:0/3255203402 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9001d070 con 0x7fbca0100dc0 2024-03-24T12:10:48.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.110+0000 7fbca48db700 1 -- 172.21.15.6:0/3255203402 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca019ba70 con 0x7fbca0100dc0 2024-03-24T12:10:48.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.110+0000 7fbca48db700 1 -- 172.21.15.6:0/3255203402 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca01a3330 con 0x7fbca0100dc0 2024-03-24T12:10:48.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.110+0000 7fbc977fe700 1 -- 172.21.15.6:0/3255203402 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fbc9000bca0 con 0x7fbca0100dc0 2024-03-24T12:10:48.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.111+0000 7fbc977fe700 1 -- 172.21.15.6:0/3255203402 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc90022bf0 con 0x7fbca0100dc0 2024-03-24T12:10:48.112 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.111+0000 7fbca48db700 1 -- 172.21.15.6:0/3255203402 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc80005120 con 0x7fbca0100dc0 2024-03-24T12:10:48.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.112+0000 7fbc977fe700 1 -- 172.21.15.6:0/3255203402 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7fbc9000b820 con 0x7fbca0100dc0 2024-03-24T12:10:48.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.112+0000 7fbc977fe700 1 --2- 172.21.15.6:0/3255203402 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fbc8c06d660 0x7fbc8c06fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:10:48.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.112+0000 7fbc977fe700 1 -- 172.21.15.6:0/3255203402 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7fbc9008e6e0 con 0x7fbca0100dc0 2024-03-24T12:10:48.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.112+0000 7fbc97fff700 1 --2- 172.21.15.6:0/3255203402 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fbc8c06d660 0x7fbc8c06fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:10:48.113 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.113+0000 7fbc97fff700 1 --2- 172.21.15.6:0/3255203402 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fbc8c06d660 0x7fbc8c06fb20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbca0100830 tx=0x7fbc88007480 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:10:48.116 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.117+0000 7fbc977fe700 1 -- 172.21.15.6:0/3255203402 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbc90057d30 con 0x7fbca0100dc0 2024-03-24T12:10:48.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.366+0000 7fbca48db700 1 -- 172.21.15.6:0/3255203402 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fbc800053d0 con 0x7fbca0100dc0 2024-03-24T12:10:48.366 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:10:48 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a[79411]: ts=2024-03-24T12:10:48.169Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000552356s 2024-03-24T12:10:48.366 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.367+0000 7fbc977fe700 1 -- 172.21.15.6:0/3255203402 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fbc9007b080 con 0x7fbca0100dc0 2024-03-24T12:10:48.367 INFO:teuthology.orchestra.run.smithi006.stdout: 2024-03-24T12:10:48.367 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-24T12:10:48.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.369+0000 7fbca48db700 1 -- 172.21.15.6:0/3255203402 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fbc8c06d660 msgr2=0x7fbc8c06fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:48.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.369+0000 7fbca48db700 1 --2- 172.21.15.6:0/3255203402 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fbc8c06d660 0x7fbc8c06fb20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbca0100830 tx=0x7fbc88007480 comp rx=0 tx=0).stop 2024-03-24T12:10:48.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.369+0000 7fbca48db700 1 -- 172.21.15.6:0/3255203402 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbca0100dc0 msgr2=0x7fbca01965e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:10:48.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.369+0000 7fbca48db700 1 --2- 172.21.15.6:0/3255203402 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbca0100dc0 0x7fbca01965e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbc900098e0 tx=0x7fbc90005740 comp rx=0 tx=0).stop 2024-03-24T12:10:48.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.369+0000 7fbca48db700 1 -- 172.21.15.6:0/3255203402 shutdown_connections 2024-03-24T12:10:48.368 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.369+0000 7fbca48db700 1 --2- 172.21.15.6:0/3255203402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbca0105e80 0x7fbca019aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:48.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.369+0000 7fbca48db700 1 --2- 172.21.15.6:0/3255203402 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fbc8c06d660 0x7fbc8c06fb20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:48.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.369+0000 7fbca48db700 1 --2- 172.21.15.6:0/3255203402 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbca0100dc0 0x7fbca01965e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:48.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.369+0000 7fbca48db700 1 --2- 172.21.15.6:0/3255203402 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbca0101770 0x7fbca0196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:10:48.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.369+0000 7fbca48db700 1 -- 172.21.15.6:0/3255203402 >> 172.21.15.6:0/3255203402 conn(0x7fbca0077ae0 msgr2=0x7fbca00fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:10:48.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.369+0000 7fbca48db700 1 -- 172.21.15.6:0/3255203402 shutdown_connections 2024-03-24T12:10:48.369 INFO:teuthology.orchestra.run.smithi006.stderr:2024-03-24T12:10:48.369+0000 7fbca48db700 1 -- 172.21.15.6:0/3255203402 wait complete. 2024-03-24T12:10:48.684 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:48 smithi006 ceph-mon[30108]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:10:48.684 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3255203402' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-24T12:10:48.685 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:48 smithi006 ceph-mon[41734]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:10:48.685 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3255203402' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-24T12:10:48.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:48 smithi090 ceph-mon[33646]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:10:48.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3255203402' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-24T12:10:49.046 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-24T12:10:49.046 INFO:tasks.cephadm:Setup complete, yielding 2024-03-24T12:10:49.046 INFO:teuthology.run_tasks:Running task workunit... 2024-03-24T12:10:49.059 INFO:tasks.workunit:Pulling workunits from ref f73f1dfd96849300c4322bd7259a3ab37f94c200 2024-03-24T12:10:49.060 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-24T12:10:49.060 DEBUG:teuthology.orchestra.run.smithi006:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-24T12:10:49.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T12:10:49.076 INFO:teuthology.orchestra.run.smithi006.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-24T12:10:49.076 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-03-24T12:10:49.131 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-03-24T12:10:49.131 DEBUG:teuthology.orchestra.run.smithi006:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-03-24T12:10:49.186 INFO:tasks.workunit:timeout=3h 2024-03-24T12:10:49.186 INFO:tasks.workunit:cleanup=True 2024-03-24T12:10:49.187 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout f73f1dfd96849300c4322bd7259a3ab37f94c200 2024-03-24T12:10:49.241 INFO:tasks.workunit.client.0.smithi006.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-24T12:10:50.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:50 smithi090 ceph-mon[33646]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:50.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:50 smithi006 ceph-mon[30108]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:50.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:50 smithi006 ceph-mon[41734]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:52.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:52 smithi090 ceph-mon[33646]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:52.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:52 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:52.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:10:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:10:52.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:52 smithi006 ceph-mon[30108]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:52.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:52 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:52.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:52 smithi006 ceph-mon[41734]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:52.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:52 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:10:53.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:53 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:10:53.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:53 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:10:53.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:53 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:10:54.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:54 smithi090 ceph-mon[33646]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:10:54.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:54 smithi006 ceph-mon[30108]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:10:54.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:54 smithi006 ceph-mon[41734]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:10:55.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:10:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:10:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:10:56.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:56 smithi090 ceph-mon[33646]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:56.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:56 smithi006 ceph-mon[30108]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:56.461 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:10:56 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a[79411]: ts=2024-03-24T12:10:56.173Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.004324097s 2024-03-24T12:10:56.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:56 smithi006 ceph-mon[41734]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:10:58.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:10:58 smithi090 ceph-mon[33646]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:10:58.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:10:58 smithi006 ceph-mon[30108]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:10:58.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:10:58 smithi006 ceph-mon[41734]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:00.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:00 smithi090 ceph-mon[33646]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:00.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:00 smithi006 ceph-mon[30108]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:00.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:00 smithi006 ceph-mon[41734]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:02.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:02 smithi090 ceph-mon[33646]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:02.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:02 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:11:02.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:11:02 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:11:02.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:02 smithi006 ceph-mon[30108]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:02.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:02 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:11:02.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:02 smithi006 ceph-mon[41734]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:02.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:02 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:11:03.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:03 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:11:03.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:03 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:11:03.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:03 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:11:04.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:04 smithi090 ceph-mon[33646]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:04.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:04 smithi006 ceph-mon[30108]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:04.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:04 smithi006 ceph-mon[41734]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:05.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:11:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:11:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:11:06.383 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:06 smithi090 ceph-mon[33646]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:06.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:06 smithi006 ceph-mon[30108]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:06.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:06 smithi006 ceph-mon[41734]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:07.928 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:07 smithi090 systemd[1]: Starting Ceph grafana.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:11:08.205 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-mon[33646]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:08.205 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:07 smithi090 podman[66658]: 2024-03-24 12:11:07.843364633 +0000 UTC m=+0.017754764 image pull a0ea5e3c46d9434bd173d585b7c2638c7451eaf2543d1d22f9e78442615aaffb quay.io/ceph/ceph-grafana:9.4.12 2024-03-24T12:11:08.205 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:07 smithi090 podman[66658]: 2024-03-24 12:11:07.962298757 +0000 UTC m=+0.136688874 container create d81656c62fe7fb48f5fdabb78de8d89df9dd4f8660236481bdb5a0ed75795d98 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a, io.openshift.expose-services=, maintainer=Paul Cuzner , com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, name=ubi8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, description=Ceph Grafana Container, architecture=x86_64, io.k8s.display-name=Red Hat Universal Base Image 8, io.buildah.version=1.32.0, com.redhat.component=ubi8-container, version=8.5, vcs-type=git, io.openshift.tags=base rhel8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., release=239.1651231664, vendor=Red Hat, Inc., build-date=2022-04-29T11:48:11.055897, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, summary=Grafana Container configured for Ceph mgr/dashboard integration, distribution-scope=public) 2024-03-24T12:11:08.457 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 podman[66658]: 2024-03-24 12:11:08.206733357 +0000 UTC m=+0.381123487 container init d81656c62fe7fb48f5fdabb78de8d89df9dd4f8660236481bdb5a0ed75795d98 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a, maintainer=Paul Cuzner , com.redhat.component=ubi8-container, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, name=ubi8, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8, distribution-scope=public, architecture=x86_64, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vendor=Red Hat, Inc., release=239.1651231664, io.openshift.expose-services=, build-date=2022-04-29T11:48:11.055897, vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., version=8.5, description=Ceph Grafana Container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.buildah.version=1.32.0) 2024-03-24T12:11:08.457 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 podman[66658]: 2024-03-24 12:11:08.21229101 +0000 UTC m=+0.386681128 container start d81656c62fe7fb48f5fdabb78de8d89df9dd4f8660236481bdb5a0ed75795d98 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a, vcs-type=git, release=239.1651231664, description=Ceph Grafana Container, vendor=Red Hat, Inc., maintainer=Paul Cuzner , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.component=ubi8-container, distribution-scope=public, io.buildah.version=1.32.0, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, architecture=x86_64, build-date=2022-04-29T11:48:11.055897, name=ubi8, version=8.5, io.openshift.expose-services=) 2024-03-24T12:11:08.457 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 bash[66658]: d81656c62fe7fb48f5fdabb78de8d89df9dd4f8660236481bdb5a0ed75795d98 2024-03-24T12:11:08.457 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 systemd[1]: Started Ceph grafana.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:11:08.457 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=settings t=2024-03-24T12:11:08.366401013Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-03-24T12:11:08.457 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=settings t=2024-03-24T12:11:08.36659862Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-03-24T12:11:08.457 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=settings t=2024-03-24T12:11:08.366616921Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-03-24T12:11:08.457 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=settings t=2024-03-24T12:11:08.366623673Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-03-24T12:11:08.457 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=settings t=2024-03-24T12:11:08.366629742Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-03-24T12:11:08.457 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=settings t=2024-03-24T12:11:08.36663569Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-03-24T12:11:08.457 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=settings t=2024-03-24T12:11:08.366641311Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-03-24T12:11:08.458 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=settings t=2024-03-24T12:11:08.36664698Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-03-24T12:11:08.458 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=settings t=2024-03-24T12:11:08.36665281Z level=info msg="Path Home" path=/usr/share/grafana 2024-03-24T12:11:08.458 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=settings t=2024-03-24T12:11:08.366658417Z level=info msg="Path Data" path=/var/lib/grafana 2024-03-24T12:11:08.458 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=settings t=2024-03-24T12:11:08.366664761Z level=info msg="Path Logs" path=/var/log/grafana 2024-03-24T12:11:08.458 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=settings t=2024-03-24T12:11:08.366670136Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-03-24T12:11:08.458 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=settings t=2024-03-24T12:11:08.366675886Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-03-24T12:11:08.458 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=settings t=2024-03-24T12:11:08.366682102Z level=info msg="App mode production" 2024-03-24T12:11:08.458 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=sqlstore t=2024-03-24T12:11:08.366729268Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-03-24T12:11:08.458 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=sqlstore t=2024-03-24T12:11:08.366746587Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-03-24T12:11:08.458 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.367650494Z level=info msg="Starting DB migrations" 2024-03-24T12:11:08.458 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.367740732Z level=info msg="Executing migration" id="create migration_log table" 2024-03-24T12:11:08.458 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.368290209Z level=info msg="Executing migration" id="create user table" 2024-03-24T12:11:08.458 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.368704166Z level=info msg="Executing migration" id="add unique index user.login" 2024-03-24T12:11:08.458 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.369140223Z level=info msg="Executing migration" id="add unique index user.email" 2024-03-24T12:11:08.459 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.369696753Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-03-24T12:11:08.459 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.370176026Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-03-24T12:11:08.459 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.370616367Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-03-24T12:11:08.459 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.371358401Z level=info msg="Executing migration" id="create user table v2" 2024-03-24T12:11:08.460 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.371783257Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-03-24T12:11:08.460 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.37219704Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-03-24T12:11:08.460 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.372664771Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-03-24T12:11:08.460 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.372990078Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-03-24T12:11:08.460 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.373351793Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-03-24T12:11:08.460 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.373736284Z level=info msg="Executing migration" id="Update user table charset" 2024-03-24T12:11:08.460 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.374019101Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-03-24T12:11:08.460 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.374459823Z level=info msg="Executing migration" id="Add missing user data" 2024-03-24T12:11:08.460 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.374798728Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-03-24T12:11:08.460 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.375244794Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-03-24T12:11:08.460 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.375673544Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-03-24T12:11:08.460 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.378379309Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-03-24T12:11:08.460 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.379565204Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-03-24T12:11:08.461 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.38123812Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-03-24T12:11:08.461 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.383869594Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-03-24T12:11:08.461 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.386764383Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-03-24T12:11:08.461 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.38943191Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-03-24T12:11:08.461 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.392039689Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-03-24T12:11:08.461 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.393026157Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-03-24T12:11:08.461 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.393868831Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-03-24T12:11:08.461 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.394637686Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-03-24T12:11:08.461 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.395534556Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-03-24T12:11:08.461 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.396050478Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-03-24T12:11:08.461 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.39668091Z level=info msg="Executing migration" id="create temp_user v2" 2024-03-24T12:11:08.461 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.397593825Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-03-24T12:11:08.461 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.398213199Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-03-24T12:11:08.461 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.398608414Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-03-24T12:11:08.462 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.399404622Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-03-24T12:11:08.462 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.399837336Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-03-24T12:11:08.462 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.400351397Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-03-24T12:11:08.462 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.401020421Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-03-24T12:11:08.462 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.401357376Z level=info msg="Executing migration" id="create star table" 2024-03-24T12:11:08.462 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.402207637Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-03-24T12:11:08.462 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.402647266Z level=info msg="Executing migration" id="create org table v1" 2024-03-24T12:11:08.462 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.403041065Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-03-24T12:11:08.462 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.403882563Z level=info msg="Executing migration" id="create org_user table v1" 2024-03-24T12:11:08.462 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.404683044Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-03-24T12:11:08.462 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.405651743Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-03-24T12:11:08.462 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.406624491Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-03-24T12:11:08.462 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.407147268Z level=info msg="Executing migration" id="Update org table charset" 2024-03-24T12:11:08.462 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.408099772Z level=info msg="Executing migration" id="Update org_user table charset" 2024-03-24T12:11:08.463 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.408867983Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-03-24T12:11:08.463 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.409861696Z level=info msg="Executing migration" id="create dashboard table" 2024-03-24T12:11:08.463 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.410355134Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-03-24T12:11:08.463 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.41133462Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-03-24T12:11:08.463 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.412071724Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-03-24T12:11:08.463 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.413041451Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-03-24T12:11:08.463 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.413522047Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-03-24T12:11:08.463 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.414325473Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-03-24T12:11:08.463 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.415211124Z level=info msg="Executing migration" id="create dashboard v2" 2024-03-24T12:11:08.463 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.415624153Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-03-24T12:11:08.463 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.416485157Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-03-24T12:11:08.463 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.416944842Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-03-24T12:11:08.463 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.419840707Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-03-24T12:11:08.463 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.422701044Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-03-24T12:11:08.464 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.425675037Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-03-24T12:11:08.464 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.428297114Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-03-24T12:11:08.464 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.428816182Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-03-24T12:11:08.464 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.431572634Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-03-24T12:11:08.464 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.434534116Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-03-24T12:11:08.464 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.437582665Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-03-24T12:11:08.464 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.440348124Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-03-24T12:11:08.464 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.443084178Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-03-24T12:11:08.464 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.443369797Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-03-24T12:11:08.464 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.446244772Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-03-24T12:11:08.464 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.446849261Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-03-24T12:11:08.464 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.449190019Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-03-24T12:11:08.465 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.452012727Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-03-24T12:11:08.465 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.45256226Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-03-24T12:11:08.465 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.455353677Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-03-24T12:11:08.465 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.455791077Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-03-24T12:11:08.465 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.458125973Z level=info msg="Executing migration" id="Update dashboard title length" 2024-03-24T12:11:08.465 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.460824692Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-03-24T12:11:08.466 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:08 smithi006 ceph-mon[30108]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:08.466 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:08 smithi006 ceph-mon[41734]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:08.707 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.463595997Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-03-24T12:11:08.707 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.466261345Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-03-24T12:11:08.707 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.469022471Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-03-24T12:11:08.707 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.47168672Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-03-24T12:11:08.707 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.472134299Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-03-24T12:11:08.708 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.475087916Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-03-24T12:11:08.708 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.477833455Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-03-24T12:11:08.708 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.480484819Z level=info msg="Executing migration" id="Add check_sum column" 2024-03-24T12:11:08.708 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.48116557Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-03-24T12:11:08.708 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.483991779Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-03-24T12:11:08.708 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.486695227Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-03-24T12:11:08.708 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.489280018Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-03-24T12:11:08.708 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.489701608Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-03-24T12:11:08.708 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.492571699Z level=info msg="Executing migration" id="create data_source table" 2024-03-24T12:11:08.708 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.495110177Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-03-24T12:11:08.708 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.495957085Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-03-24T12:11:08.708 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.496380323Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-03-24T12:11:08.708 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.497187364Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-03-24T12:11:08.709 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.497579771Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-03-24T12:11:08.709 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.498588453Z level=info msg="Executing migration" id="create data_source table v2" 2024-03-24T12:11:08.709 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.499352626Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-03-24T12:11:08.709 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.499773116Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-03-24T12:11:08.709 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.500637919Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-03-24T12:11:08.709 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.501601795Z level=info msg="Executing migration" id="Add column with_credentials" 2024-03-24T12:11:08.709 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.502332664Z level=info msg="Executing migration" id="Add secure json data column" 2024-03-24T12:11:08.709 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.502883599Z level=info msg="Executing migration" id="Update data_source table charset" 2024-03-24T12:11:08.709 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.503834937Z level=info msg="Executing migration" id="Update initial version to 1" 2024-03-24T12:11:08.710 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.504672332Z level=info msg="Executing migration" id="Add read_only data column" 2024-03-24T12:11:08.710 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.5054128Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-03-24T12:11:08.710 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.505719133Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-03-24T12:11:08.710 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.506476313Z level=info msg="Executing migration" id="Add uid column" 2024-03-24T12:11:08.710 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.507021527Z level=info msg="Executing migration" id="Update uid value" 2024-03-24T12:11:08.710 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.50734856Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-03-24T12:11:08.710 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.508292198Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-03-24T12:11:08.710 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.508924817Z level=info msg="Executing migration" id="create api_key table" 2024-03-24T12:11:08.710 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.509833866Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-03-24T12:11:08.710 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.510641117Z level=info msg="Executing migration" id="add index api_key.key" 2024-03-24T12:11:08.710 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.511587976Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-03-24T12:11:08.710 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.512213166Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-03-24T12:11:08.710 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.513024044Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-03-24T12:11:08.710 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.513879161Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-03-24T12:11:08.711 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.514767868Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-03-24T12:11:08.711 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.515906184Z level=info msg="Executing migration" id="create api_key table v2" 2024-03-24T12:11:08.711 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.516866637Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-03-24T12:11:08.711 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.517748814Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-03-24T12:11:08.711 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.518483577Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-03-24T12:11:08.711 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.519480688Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-03-24T12:11:08.711 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.520190012Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-03-24T12:11:08.711 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.521207959Z level=info msg="Executing migration" id="Update api_key table charset" 2024-03-24T12:11:08.711 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.521981574Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-03-24T12:11:08.711 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.525009399Z level=info msg="Executing migration" id="Add service account foreign key" 2024-03-24T12:11:08.711 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.527467359Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-03-24T12:11:08.711 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.530337279Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-03-24T12:11:08.711 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.533088547Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-03-24T12:11:08.711 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.536186225Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-03-24T12:11:08.711 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.538880442Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-03-24T12:11:08.712 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.541823428Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-03-24T12:11:08.712 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.544681849Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-03-24T12:11:08.712 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.547471206Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-03-24T12:11:08.713 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.550260652Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-03-24T12:11:08.713 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.553112678Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-03-24T12:11:08.713 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.555823931Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-03-24T12:11:08.713 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.558594564Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-03-24T12:11:08.713 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.561412576Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-03-24T12:11:08.713 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.564165464Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-03-24T12:11:08.713 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.567093813Z level=info msg="Executing migration" id="create quota table v1" 2024-03-24T12:11:08.713 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.567512443Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-03-24T12:11:08.713 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.570230628Z level=info msg="Executing migration" id="Update quota table charset" 2024-03-24T12:11:08.713 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.573156112Z level=info msg="Executing migration" id="create plugin_setting table" 2024-03-24T12:11:08.713 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.573578967Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-03-24T12:11:08.713 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.57635215Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-03-24T12:11:08.713 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.579101844Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-03-24T12:11:08.713 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.581951896Z level=info msg="Executing migration" id="create session table" 2024-03-24T12:11:08.714 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.582640306Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-03-24T12:11:08.714 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.582940526Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-03-24T12:11:08.714 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.585270071Z level=info msg="Executing migration" id="create playlist table v2" 2024-03-24T12:11:08.714 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.585728936Z level=info msg="Executing migration" id="create playlist item table v2" 2024-03-24T12:11:08.714 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.588557919Z level=info msg="Executing migration" id="Update playlist table charset" 2024-03-24T12:11:08.714 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.591574417Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-03-24T12:11:08.714 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.591918284Z level=info msg="Executing migration" id="drop preferences table v2" 2024-03-24T12:11:08.714 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.592225359Z level=info msg="Executing migration" id="drop preferences table v3" 2024-03-24T12:11:08.714 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.592572608Z level=info msg="Executing migration" id="create preferences table v3" 2024-03-24T12:11:08.714 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.593006523Z level=info msg="Executing migration" id="Update preferences table charset" 2024-03-24T12:11:08.714 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.595872055Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-03-24T12:11:08.714 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.596713491Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-03-24T12:11:08.715 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.597009185Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-03-24T12:11:08.715 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.59953425Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-03-24T12:11:08.715 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.602457104Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-03-24T12:11:08.715 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.605093571Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-03-24T12:11:08.715 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.605779584Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-03-24T12:11:08.715 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.608758331Z level=info msg="Executing migration" id="create alert table v1" 2024-03-24T12:11:08.715 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.611264054Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-03-24T12:11:08.715 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.61416401Z level=info msg="Executing migration" id="add index alert state" 2024-03-24T12:11:08.715 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.616790695Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-03-24T12:11:08.715 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.617252155Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-03-24T12:11:08.715 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.617630431Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-03-24T12:11:08.715 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.618062099Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-03-24T12:11:08.715 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.618558853Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-03-24T12:11:08.715 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.619989535Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-03-24T12:11:08.716 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.620410356Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-03-24T12:11:08.716 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.620874351Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-03-24T12:11:08.716 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.62124176Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-03-24T12:11:08.716 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.621608006Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-03-24T12:11:08.716 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.62201667Z level=info msg="Executing migration" id="Add column is_default" 2024-03-24T12:11:08.716 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.622676962Z level=info msg="Executing migration" id="Add column frequency" 2024-03-24T12:11:08.716 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.623308929Z level=info msg="Executing migration" id="Add column send_reminder" 2024-03-24T12:11:08.716 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.623980592Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-03-24T12:11:08.716 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.624643219Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-03-24T12:11:08.716 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.62509254Z level=info msg="Executing migration" id="Update alert table charset" 2024-03-24T12:11:08.716 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.625405882Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-03-24T12:11:08.716 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.62571359Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-03-24T12:11:08.716 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.626142924Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-03-24T12:11:08.716 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.626636231Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-03-24T12:11:08.716 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.627004976Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-03-24T12:11:08.717 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.627439495Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-03-24T12:11:08.717 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.627863353Z level=info msg="Executing migration" id="Add for to alert table" 2024-03-24T12:11:08.717 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.628532628Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-03-24T12:11:08.717 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.629180227Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-03-24T12:11:08.717 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.629528627Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-03-24T12:11:08.717 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.630000553Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-03-24T12:11:08.717 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.630456113Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-03-24T12:11:08.717 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.631104644Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-03-24T12:11:08.717 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.631440045Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-03-24T12:11:08.717 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.631861504Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-03-24T12:11:08.717 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.632303096Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-03-24T12:11:08.717 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.632636534Z level=info msg="Executing migration" id="create annotation table v5" 2024-03-24T12:11:08.717 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.633090957Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-03-24T12:11:08.717 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.635694955Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-03-24T12:11:08.717 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.636185631Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-03-24T12:11:08.718 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.636609507Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-03-24T12:11:08.718 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.63707622Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-03-24T12:11:08.718 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.639824616Z level=info msg="Executing migration" id="Update annotation table charset" 2024-03-24T12:11:08.718 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.642792991Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-03-24T12:11:08.718 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.643496553Z level=info msg="Executing migration" id="Drop category_id index" 2024-03-24T12:11:08.718 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.646338067Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-03-24T12:11:08.718 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.649048072Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-03-24T12:11:08.718 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.6517031Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-03-24T12:11:08.718 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.654464235Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-03-24T12:11:08.718 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.657095995Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-03-24T12:11:08.718 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.660001839Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-03-24T12:11:08.718 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.662603917Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-03-24T12:11:08.718 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.663067099Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-03-24T12:11:08.718 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.665815706Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-03-24T12:11:08.718 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.66660053Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-03-24T12:11:08.719 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.667482574Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-03-24T12:11:08.719 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.668179466Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-03-24T12:11:08.719 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.669005121Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-03-24T12:11:08.719 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.669874409Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-03-24T12:11:08.719 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.67090617Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-03-24T12:11:08.719 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.671276955Z level=info msg="Executing migration" id="Add epoch_end column" 2024-03-24T12:11:08.719 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.672069908Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-03-24T12:11:08.719 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.672496118Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-03-24T12:11:08.719 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.672875143Z level=info msg="Executing migration" id="Move region to single row" 2024-03-24T12:11:08.720 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.673243776Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-03-24T12:11:08.720 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.673681831Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-03-24T12:11:08.720 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.674053853Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-03-24T12:11:08.720 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.674474758Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-03-24T12:11:08.720 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.674937829Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-03-24T12:11:08.721 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.675345669Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-03-24T12:11:08.721 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.675778142Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-03-24T12:11:08.721 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.676097065Z level=info msg="Executing migration" id="create test_data table" 2024-03-24T12:11:08.721 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.676505239Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-03-24T12:11:08.721 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.676887704Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-03-24T12:11:08.721 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.677314228Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-03-24T12:11:08.721 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.67828856Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-03-24T12:11:08.721 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.679019169Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-03-24T12:11:08.721 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.679956048Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-03-24T12:11:08.721 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.680364105Z level=info msg="Executing migration" id="create team table" 2024-03-24T12:11:08.721 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.681222217Z level=info msg="Executing migration" id="add index team.org_id" 2024-03-24T12:11:08.721 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.682184588Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-03-24T12:11:08.721 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.683090855Z level=info msg="Executing migration" id="create team member table" 2024-03-24T12:11:08.721 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.683636423Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-03-24T12:11:08.721 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.684501889Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-03-24T12:11:08.722 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.685382045Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-03-24T12:11:08.722 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.686211274Z level=info msg="Executing migration" id="Add column email to team table" 2024-03-24T12:11:08.722 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.686922453Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-03-24T12:11:08.722 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.68778015Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-03-24T12:11:08.722 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.688657386Z level=info msg="Executing migration" id="create dashboard acl table" 2024-03-24T12:11:08.722 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.689532247Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-03-24T12:11:08.722 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.690009942Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-03-24T12:11:08.722 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.693007497Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-03-24T12:11:08.722 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.693561014Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-03-24T12:11:08.722 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.696531074Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-03-24T12:11:08.722 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.697085624Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-03-24T12:11:08.722 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.699898712Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-03-24T12:11:08.722 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.702725799Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-03-24T12:11:08.722 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.705663365Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-03-24T12:11:08.722 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.708392097Z level=info msg="Executing migration" id="create tag table" 2024-03-24T12:11:08.723 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.711136637Z level=info msg="Executing migration" id="add index tag.key_value" 2024-03-24T12:11:08.723 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.713818985Z level=info msg="Executing migration" id="create login attempt table" 2024-03-24T12:11:08.957 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.716594575Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-03-24T12:11:08.958 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.719408726Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-03-24T12:11:08.958 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.722129919Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-03-24T12:11:08.958 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.725077947Z level=info msg="Executing migration" id="create login_attempt v2" 2024-03-24T12:11:08.958 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.727743961Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-03-24T12:11:08.958 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.730730549Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-03-24T12:11:08.958 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.733517839Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-03-24T12:11:08.958 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.736299056Z level=info msg="Executing migration" id="create user auth table" 2024-03-24T12:11:08.958 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.739120599Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-03-24T12:11:08.958 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.741787091Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-03-24T12:11:08.958 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.744533626Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-03-24T12:11:08.959 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.747330167Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-03-24T12:11:08.959 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.750019916Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-03-24T12:11:08.959 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.752750212Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-03-24T12:11:08.959 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.755658743Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-03-24T12:11:08.959 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.758198639Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-03-24T12:11:08.959 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.761038733Z level=info msg="Executing migration" id="create server_lock table" 2024-03-24T12:11:08.959 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.761811555Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-03-24T12:11:08.959 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.764841766Z level=info msg="Executing migration" id="create user auth token table" 2024-03-24T12:11:08.960 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.767441361Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-03-24T12:11:08.960 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.770116021Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-03-24T12:11:08.960 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.772667512Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-03-24T12:11:08.960 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.775511927Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-03-24T12:11:08.960 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.778218846Z level=info msg="Executing migration" id="create cache_data table" 2024-03-24T12:11:08.960 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.780941876Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-03-24T12:11:08.960 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.783527634Z level=info msg="Executing migration" id="create short_url table v1" 2024-03-24T12:11:08.960 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.7862716Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-03-24T12:11:08.961 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.788894449Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-03-24T12:11:08.961 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.791742995Z level=info msg="Executing migration" id="delete alert_definition table" 2024-03-24T12:11:08.961 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.794221444Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-03-24T12:11:08.961 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.797127404Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-03-24T12:11:08.961 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.799834118Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-03-24T12:11:08.961 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.802554007Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-03-24T12:11:08.961 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.805087929Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-03-24T12:11:08.961 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.807697061Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-03-24T12:11:08.961 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.808424794Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-03-24T12:11:08.961 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.809389298Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-03-24T12:11:08.961 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.810151465Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-03-24T12:11:08.961 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.811026984Z level=info msg="Executing migration" id="drop alert_definition table" 2024-03-24T12:11:08.961 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.811878248Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-03-24T12:11:08.962 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.812696199Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-03-24T12:11:08.962 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.813640192Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-03-24T12:11:08.962 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.814411964Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-03-24T12:11:08.962 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.815416337Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-03-24T12:11:08.962 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.816106489Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-03-24T12:11:08.962 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.817128893Z level=info msg="Executing migration" id="create alert_instance table" 2024-03-24T12:11:08.962 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.817822352Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-03-24T12:11:08.963 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.818656065Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-03-24T12:11:08.963 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.819515578Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-03-24T12:11:08.963 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.820352395Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-03-24T12:11:08.963 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.821172779Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-03-24T12:11:08.963 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.822169407Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-03-24T12:11:08.963 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.825529816Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-03-24T12:11:08.963 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.828856098Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-03-24T12:11:08.963 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.829867761Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-03-24T12:11:08.963 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.830613195Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-03-24T12:11:08.963 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.831504938Z level=info msg="Executing migration" id="create alert_rule table" 2024-03-24T12:11:08.963 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.83236044Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-03-24T12:11:08.963 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.833361481Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-03-24T12:11:08.963 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.834238301Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-03-24T12:11:08.964 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.834932694Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-03-24T12:11:08.964 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.835791723Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-03-24T12:11:08.964 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.836656113Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-03-24T12:11:08.964 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.837501615Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-03-24T12:11:08.964 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.838384267Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-03-24T12:11:08.964 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.839337353Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-03-24T12:11:08.964 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.840218069Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-03-24T12:11:08.964 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.841147918Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-03-24T12:11:08.964 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.842007622Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-03-24T12:11:08.964 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.84300057Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-03-24T12:11:08.964 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.84589997Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-03-24T12:11:08.964 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.848808373Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-03-24T12:11:08.964 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.851912734Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-03-24T12:11:08.965 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.854828084Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-03-24T12:11:08.965 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.857878705Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-03-24T12:11:08.965 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.860579928Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-03-24T12:11:08.965 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.863573753Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-03-24T12:11:08.965 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.8664709Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-03-24T12:11:08.965 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.869490003Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-03-24T12:11:08.965 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.872465491Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-03-24T12:11:08.965 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.875462386Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-03-24T12:11:08.965 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.878401611Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-03-24T12:11:08.965 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.881233519Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-03-24T12:11:08.965 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.884298839Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-03-24T12:11:08.965 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.887109528Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-03-24T12:11:08.966 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.890045054Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-03-24T12:11:08.966 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.893144433Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-03-24T12:11:08.966 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.896014995Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-03-24T12:11:08.966 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.898709479Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-03-24T12:11:08.966 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.901363147Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-03-24T12:11:08.966 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.904109763Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-03-24T12:11:08.968 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.906871958Z level=info msg="Executing migration" id="create provenance_type table" 2024-03-24T12:11:08.968 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.909646723Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-03-24T12:11:08.968 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.912189389Z level=info msg="Executing migration" id="create alert_image table" 2024-03-24T12:11:08.969 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.914890559Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-03-24T12:11:08.969 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.917676903Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-03-24T12:11:08.969 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.92045175Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-03-24T12:11:08.969 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.923265709Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-03-24T12:11:08.969 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.925899258Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-03-24T12:11:08.969 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.925991827Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-03-24T12:11:08.969 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.92864256Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-03-24T12:11:08.969 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.931153573Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-03-24T12:11:08.969 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.934038818Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-03-24T12:11:08.969 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.934162304Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-03-24T12:11:08.969 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.934277424Z level=info msg="alerts found to migrate" alerts=0 2024-03-24T12:11:08.969 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.936760929Z level=info msg="Executing migration" id="create library_element table v1" 2024-03-24T12:11:08.969 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.939646029Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-03-24T12:11:08.970 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.942262894Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-03-24T12:11:08.970 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.943028137Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-03-24T12:11:08.970 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.94381082Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-03-24T12:11:08.970 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.94478047Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-03-24T12:11:08.970 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.945461719Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-03-24T12:11:08.970 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.946458182Z level=info msg="Executing migration" id="create data_keys table" 2024-03-24T12:11:08.970 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.947107326Z level=info msg="Executing migration" id="create secrets table" 2024-03-24T12:11:08.970 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.94812726Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-03-24T12:11:08.970 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.952581383Z level=info msg="Executing migration" id="add name column into data_keys" 2024-03-24T12:11:08.970 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.953551679Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-03-24T12:11:08.970 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.954425876Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-03-24T12:11:08.970 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.958675783Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-03-24T12:11:08.970 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.963053506Z level=info msg="Executing migration" id="create kv_store table v1" 2024-03-24T12:11:09.354 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.963854315Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-03-24T12:11:09.354 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.964694401Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-03-24T12:11:09.354 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.965536342Z level=info msg="Executing migration" id="create permission table" 2024-03-24T12:11:09.354 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.966330537Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-03-24T12:11:09.355 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.967161208Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-03-24T12:11:09.355 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.968013902Z level=info msg="Executing migration" id="create role table" 2024-03-24T12:11:09.355 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.968917202Z level=info msg="Executing migration" id="add column display_name" 2024-03-24T12:11:09.355 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.969853577Z level=info msg="Executing migration" id="add column group_name" 2024-03-24T12:11:09.355 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.970763383Z level=info msg="Executing migration" id="add index role.org_id" 2024-03-24T12:11:09.355 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.971604857Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-03-24T12:11:09.355 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.972625761Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-03-24T12:11:09.355 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.973358315Z level=info msg="Executing migration" id="create team role table" 2024-03-24T12:11:09.355 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.974157188Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-03-24T12:11:09.355 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.97504615Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-03-24T12:11:09.355 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.975976057Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-03-24T12:11:09.355 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.976797131Z level=info msg="Executing migration" id="create user role table" 2024-03-24T12:11:09.355 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.977690191Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-03-24T12:11:09.355 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.978643664Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-03-24T12:11:09.356 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.979493619Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-03-24T12:11:09.356 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.980224659Z level=info msg="Executing migration" id="create builtin role table" 2024-03-24T12:11:09.356 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.981081481Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-03-24T12:11:09.356 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.981985464Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-03-24T12:11:09.356 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.982846781Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-03-24T12:11:09.356 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.985994888Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-03-24T12:11:09.357 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.988627724Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-03-24T12:11:09.357 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.991528884Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-03-24T12:11:09.357 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.994362147Z level=info msg="Executing migration" id="add unique index role.uid" 2024-03-24T12:11:09.357 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.997229758Z level=info msg="Executing migration" id="create seed assignment table" 2024-03-24T12:11:09.357 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.997689071Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-03-24T12:11:09.357 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.998160493Z level=info msg="Executing migration" id="add column hidden to role table" 2024-03-24T12:11:09.357 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.999122881Z level=info msg="Executing migration" id="create query_history table v1" 2024-03-24T12:11:09.357 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:08 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.999538489Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-03-24T12:11:09.357 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:08.999978836Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-03-24T12:11:09.357 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.000278569Z level=info msg="Executing migration" id="teams permissions migration" 2024-03-24T12:11:09.357 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.000669388Z level=info msg="Executing migration" id="dashboard permissions" 2024-03-24T12:11:09.357 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.001168327Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-03-24T12:11:09.357 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.001539504Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-03-24T12:11:09.357 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.001826742Z level=info msg="Executing migration" id="alerting notification permissions" 2024-03-24T12:11:09.358 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.002189712Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-03-24T12:11:09.358 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.002595027Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-03-24T12:11:09.358 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.003035912Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-03-24T12:11:09.358 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.003988955Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-03-24T12:11:09.358 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.004298552Z level=info msg="Executing migration" id="create correlation table v1" 2024-03-24T12:11:09.358 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.004708962Z level=info msg="Executing migration" id="add index correlations.uid" 2024-03-24T12:11:09.358 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.005152787Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-03-24T12:11:09.358 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.005581444Z level=info msg="Executing migration" id="add correlation config column" 2024-03-24T12:11:09.358 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.006580899Z level=info msg="Executing migration" id="create entity_events table" 2024-03-24T12:11:09.358 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.007016011Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-03-24T12:11:09.358 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.009766778Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-03-24T12:11:09.358 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.009944585Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-03-24T12:11:09.358 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.010291556Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-03-24T12:11:09.358 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.010384689Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-03-24T12:11:09.358 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.010660416Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-03-24T12:11:09.359 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.011207278Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-03-24T12:11:09.359 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.011619289Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-03-24T12:11:09.359 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.012045113Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-03-24T12:11:09.359 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.01246406Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-03-24T12:11:09.359 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.012837952Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-03-24T12:11:09.359 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.013256272Z level=info msg="Executing migration" id="Drop public config table" 2024-03-24T12:11:09.359 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.013637522Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-03-24T12:11:09.359 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.014038397Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-03-24T12:11:09.359 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.014490786Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-03-24T12:11:09.359 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.014906881Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-03-24T12:11:09.359 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.015352517Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-03-24T12:11:09.359 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.018302144Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-03-24T12:11:09.359 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.019290835Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-03-24T12:11:09.359 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.020271451Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-03-24T12:11:09.359 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.020576462Z level=info msg="Executing migration" id="add share column" 2024-03-24T12:11:09.360 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.021603801Z level=info msg="Executing migration" id="create default alerting folders" 2024-03-24T12:11:09.360 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.02208339Z level=info msg="Executing migration" id="create file table" 2024-03-24T12:11:09.360 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.022552159Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-03-24T12:11:09.360 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.023024426Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-03-24T12:11:09.360 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.025319537Z level=info msg="Executing migration" id="create file_meta table" 2024-03-24T12:11:09.360 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.027987627Z level=info msg="Executing migration" id="file table idx: path key" 2024-03-24T12:11:09.360 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.030738878Z level=info msg="Executing migration" id="set path collation in file table" 2024-03-24T12:11:09.360 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.033588052Z level=info msg="Executing migration" id="managed permissions migration" 2024-03-24T12:11:09.361 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.036365551Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-03-24T12:11:09.361 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.03909189Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-03-24T12:11:09.361 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.039737513Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-03-24T12:11:09.361 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.040788788Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-03-24T12:11:09.361 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.041623978Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-03-24T12:11:09.361 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.042618218Z level=info msg="Executing migration" id="update group index for alert rules" 2024-03-24T12:11:09.361 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.043480511Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-03-24T12:11:09.361 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.044233029Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-03-24T12:11:09.362 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.045189609Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-03-24T12:11:09.362 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.046359372Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-03-24T12:11:09.362 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.047403257Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-03-24T12:11:09.362 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.048290479Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-03-24T12:11:09.362 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.058132172Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-03-24T12:11:09.362 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.059003816Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-03-24T12:11:09.362 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.059920746Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-03-24T12:11:09.362 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.063449649Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-03-24T12:11:09.362 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.064294343Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-03-24T12:11:09.362 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.065151862Z level=info msg="Executing migration" id="create folder table" 2024-03-24T12:11:09.362 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.066140133Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-03-24T12:11:09.362 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.066945229Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-03-24T12:11:09.362 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.06793078Z level=info msg="Executing migration" id="Update folder title length" 2024-03-24T12:11:09.362 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.068690977Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-03-24T12:11:09.363 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=migrator t=2024-03-24T12:11:09.06958097Z level=info msg="migrations completed" performed=485 skipped=0 duration=701.859733ms 2024-03-24T12:11:09.363 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=sqlstore t=2024-03-24T12:11:09.06998083Z level=info msg="Created default organization" 2024-03-24T12:11:09.363 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=plugin.loader t=2024-03-24T12:11:09.127538739Z level=info msg="Plugin registered" pluginID=input 2024-03-24T12:11:09.363 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=plugin.loader t=2024-03-24T12:11:09.154546414Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-03-24T12:11:09.363 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=plugin.loader t=2024-03-24T12:11:09.154563543Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-03-24T12:11:09.363 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=secrets t=2024-03-24T12:11:09.15459581Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-03-24T12:11:09.363 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=query_data t=2024-03-24T12:11:09.156696648Z level=info msg="Query Service initialization" 2024-03-24T12:11:09.363 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=live.push_http t=2024-03-24T12:11:09.159031863Z level=info msg="Live Push Gateway initialization" 2024-03-24T12:11:09.617 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:09 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:09.617 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:09 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:09.617 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:09 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:09.617 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:09 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:09.617 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:09 smithi006 ceph-mon[30108]: Deploying daemon node-exporter.a on smithi006 2024-03-24T12:11:09.618 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:09 smithi006 systemd[1]: Starting Ceph node-exporter.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:11:09.618 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:09 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:09.618 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:09 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:09.618 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:09 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:09.618 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:09 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:09.618 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:09 smithi006 ceph-mon[41734]: Deploying daemon node-exporter.a on smithi006 2024-03-24T12:11:09.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:09.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:09.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:09.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:09.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-mon[33646]: Deploying daemon node-exporter.a on smithi006 2024-03-24T12:11:09.961 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:09 smithi006 bash[79958]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-03-24T12:11:10.178 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=infra.usagestats.collector t=2024-03-24T12:11:09.762676026Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-03-24T12:11:10.178 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=provisioning.datasources t=2024-03-24T12:11:09.763670302Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-03-24T12:11:10.178 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=provisioning.datasources t=2024-03-24T12:11:09.774751717Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-03-24T12:11:10.178 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=provisioning.alerting t=2024-03-24T12:11:09.785765993Z level=info msg="starting to provision alerting" 2024-03-24T12:11:10.179 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=provisioning.alerting t=2024-03-24T12:11:09.78578462Z level=info msg="finished to provision alerting" 2024-03-24T12:11:10.179 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=ngalert.state.manager t=2024-03-24T12:11:09.785865069Z level=info msg="Warming state cache for startup" 2024-03-24T12:11:10.179 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=grafanaStorageLogger t=2024-03-24T12:11:09.785879226Z level=info msg="storage starting" 2024-03-24T12:11:10.179 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=http.server t=2024-03-24T12:11:09.793400688Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-03-24T12:11:10.179 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=ngalert.state.manager t=2024-03-24T12:11:09.798726978Z level=info msg="State cache has been initialized" states=0 duration=12.861755ms 2024-03-24T12:11:10.179 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=ngalert.multiorg.alertmanager t=2024-03-24T12:11:09.798767644Z level=info msg="starting MultiOrg Alertmanager" 2024-03-24T12:11:10.179 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=ticker t=2024-03-24T12:11:09.798802346Z level=info msg=starting first_tick=2024-03-24T12:11:10Z 2024-03-24T12:11:10.179 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=sqlstore.transactions t=2024-03-24T12:11:09.815328031Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-03-24T12:11:10.179 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:11:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=sqlstore.transactions t=2024-03-24T12:11:09.831067234Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-03-24T12:11:10.436 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:10 smithi006 bash[79958]: Getting image source signatures 2024-03-24T12:11:10.436 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:10 smithi006 bash[79958]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-03-24T12:11:10.436 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:10 smithi006 bash[79958]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-03-24T12:11:10.436 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:10 smithi006 bash[79958]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-03-24T12:11:10.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:10 smithi006 ceph-mon[30108]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:10.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:10 smithi006 ceph-mon[41734]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:10.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:10 smithi090 ceph-mon[33646]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:11.962 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:11 smithi006 bash[79958]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-03-24T12:11:11.962 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:11 smithi006 bash[79958]: Writing manifest to image destination 2024-03-24T12:11:12.309 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-mon[30108]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:12.309 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:12.309 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-mon[41734]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:12.309 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:12.309 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 podman[79958]: 2024-03-24 12:11:11.95110802 +0000 UTC m=+2.429282069 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-03-24T12:11:12.309 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 podman[79958]: 2024-03-24 12:11:12.05798061 +0000 UTC m=+2.536154652 container create 77aa8ce8cb41f28d854c4fae4af2e4aaea01773712e4ad32349a9c2d4e991bad (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-24T12:11:12.310 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 podman[79958]: 2024-03-24 12:11:12.310507007 +0000 UTC m=+2.788681049 container init 77aa8ce8cb41f28d854c4fae4af2e4aaea01773712e4ad32349a9c2d4e991bad (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-24T12:11:12.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:12 smithi090 ceph-mon[33646]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:12.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:12 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:12.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:11:12 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:11:12.711 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 podman[79958]: 2024-03-24 12:11:12.316585778 +0000 UTC m=+2.794759821 container start 77aa8ce8cb41f28d854c4fae4af2e4aaea01773712e4ad32349a9c2d4e991bad (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-24T12:11:12.711 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.321Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-03-24T12:11:12.711 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.321Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-03-24T12:11:12.711 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.321Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-03-24T12:11:12.711 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.321Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-03-24T12:11:12.712 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-03-24T12:11:12.712 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-03-24T12:11:12.712 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-03-24T12:11:12.712 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=node_exporter.go:117 level=info collector=arp 2024-03-24T12:11:12.712 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=node_exporter.go:117 level=info collector=bcache 2024-03-24T12:11:12.712 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=node_exporter.go:117 level=info collector=bonding 2024-03-24T12:11:12.712 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=node_exporter.go:117 level=info collector=btrfs 2024-03-24T12:11:12.712 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=node_exporter.go:117 level=info collector=conntrack 2024-03-24T12:11:12.712 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=node_exporter.go:117 level=info collector=cpu 2024-03-24T12:11:12.712 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-03-24T12:11:12.712 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=node_exporter.go:117 level=info collector=diskstats 2024-03-24T12:11:12.712 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=node_exporter.go:117 level=info collector=dmi 2024-03-24T12:11:12.713 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=node_exporter.go:117 level=info collector=edac 2024-03-24T12:11:12.713 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=node_exporter.go:117 level=info collector=entropy 2024-03-24T12:11:12.713 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-03-24T12:11:12.713 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=node_exporter.go:117 level=info collector=filefd 2024-03-24T12:11:12.713 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=node_exporter.go:117 level=info collector=filesystem 2024-03-24T12:11:12.713 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=node_exporter.go:117 level=info collector=hwmon 2024-03-24T12:11:12.713 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=node_exporter.go:117 level=info collector=infiniband 2024-03-24T12:11:12.713 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=node_exporter.go:117 level=info collector=ipvs 2024-03-24T12:11:12.714 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=node_exporter.go:117 level=info collector=loadavg 2024-03-24T12:11:12.714 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=node_exporter.go:117 level=info collector=mdadm 2024-03-24T12:11:12.714 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=node_exporter.go:117 level=info collector=meminfo 2024-03-24T12:11:12.714 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=node_exporter.go:117 level=info collector=netclass 2024-03-24T12:11:12.714 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.322Z caller=node_exporter.go:117 level=info collector=netdev 2024-03-24T12:11:12.714 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=node_exporter.go:117 level=info collector=netstat 2024-03-24T12:11:12.714 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=node_exporter.go:117 level=info collector=nfs 2024-03-24T12:11:12.714 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=node_exporter.go:117 level=info collector=nfsd 2024-03-24T12:11:12.714 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=node_exporter.go:117 level=info collector=nvme 2024-03-24T12:11:12.714 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=node_exporter.go:117 level=info collector=os 2024-03-24T12:11:12.714 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-03-24T12:11:12.714 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=node_exporter.go:117 level=info collector=pressure 2024-03-24T12:11:12.714 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=node_exporter.go:117 level=info collector=rapl 2024-03-24T12:11:12.714 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=node_exporter.go:117 level=info collector=schedstat 2024-03-24T12:11:12.714 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=node_exporter.go:117 level=info collector=selinux 2024-03-24T12:11:12.715 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=node_exporter.go:117 level=info collector=sockstat 2024-03-24T12:11:12.715 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=node_exporter.go:117 level=info collector=softnet 2024-03-24T12:11:12.715 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=node_exporter.go:117 level=info collector=stat 2024-03-24T12:11:12.715 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=node_exporter.go:117 level=info collector=tapestats 2024-03-24T12:11:12.715 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=node_exporter.go:117 level=info collector=textfile 2024-03-24T12:11:12.715 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-03-24T12:11:12.715 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=node_exporter.go:117 level=info collector=time 2024-03-24T12:11:12.715 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-03-24T12:11:12.715 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=node_exporter.go:117 level=info collector=uname 2024-03-24T12:11:12.715 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=node_exporter.go:117 level=info collector=vmstat 2024-03-24T12:11:12.715 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=node_exporter.go:117 level=info collector=xfs 2024-03-24T12:11:12.715 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=node_exporter.go:117 level=info collector=zfs 2024-03-24T12:11:12.715 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-03-24T12:11:12.715 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a[80007]: ts=2024-03-24T12:11:12.323Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-03-24T12:11:12.715 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 bash[79958]: 77aa8ce8cb41f28d854c4fae4af2e4aaea01773712e4ad32349a9c2d4e991bad 2024-03-24T12:11:12.716 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:11:12 smithi006 systemd[1]: Started Ceph node-exporter.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:11:13.421 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:13 smithi090 systemd[1]: Starting Ceph node-exporter.b for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:11:13.678 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:13 smithi090 bash[66955]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-03-24T12:11:13.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:13 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:11:13.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:13 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:13.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:13 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:13.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:13 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:13.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:13 smithi090 ceph-mon[33646]: Deploying daemon node-exporter.b on smithi090 2024-03-24T12:11:13.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:13 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:11:13.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:13 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:13.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:13 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:13.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:13 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:13.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:13 smithi006 ceph-mon[30108]: Deploying daemon node-exporter.b on smithi090 2024-03-24T12:11:13.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:13 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:11:13.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:13 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:13.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:13 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:13.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:13 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:13.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:13 smithi006 ceph-mon[41734]: Deploying daemon node-exporter.b on smithi090 2024-03-24T12:11:14.428 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:14 smithi090 bash[66955]: Getting image source signatures 2024-03-24T12:11:14.429 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:14 smithi090 bash[66955]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-03-24T12:11:14.429 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:14 smithi090 bash[66955]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-03-24T12:11:14.429 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:14 smithi090 bash[66955]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-03-24T12:11:14.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:14 smithi090 ceph-mon[33646]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:14.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:14 smithi006 ceph-mon[30108]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:14.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:14 smithi006 ceph-mon[41734]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:15.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:11:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:11:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:11:16.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:16 smithi090 ceph-mon[33646]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:16.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:16 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:11:16.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:16 smithi006 ceph-mon[30108]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:16.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:16 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:11:16.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:16 smithi006 ceph-mon[41734]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:16.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:16 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:11:18.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:18 smithi090 ceph-mon[33646]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:18.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:18 smithi006 ceph-mon[30108]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:18.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:18 smithi006 ceph-mon[41734]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:20.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:20 smithi090 ceph-mon[33646]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:20.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:20 smithi006 ceph-mon[30108]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:20.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:20 smithi006 ceph-mon[41734]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:22.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:11:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:11:22.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:22 smithi090 ceph-mon[33646]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:22.928 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:22 smithi090 bash[66955]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-03-24T12:11:22.964 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:22 smithi006 ceph-mon[30108]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:22.965 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:22 smithi006 ceph-mon[41734]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:23.874 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:23 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:11:23.874 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:23 smithi090 bash[66955]: Writing manifest to image destination 2024-03-24T12:11:23.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:23 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:11:23.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:23 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:11:24.172 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:23 smithi090 podman[66955]: 2024-03-24 12:11:23.821027064 +0000 UTC m=+10.296769939 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-03-24T12:11:24.172 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:23 smithi090 podman[66955]: 2024-03-24 12:11:23.946112097 +0000 UTC m=+10.421854965 container create f2c2d5ff5c48d84f23206dd7156f71519e0a07714ec57a30e5a6fa592a68e0df (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-24T12:11:24.172 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 podman[66955]: 2024-03-24 12:11:24.173558098 +0000 UTC m=+10.649300985 container init f2c2d5ff5c48d84f23206dd7156f71519e0a07714ec57a30e5a6fa592a68e0df (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-24T12:11:24.428 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 podman[66955]: 2024-03-24 12:11:24.180245753 +0000 UTC m=+10.655988630 container start f2c2d5ff5c48d84f23206dd7156f71519e0a07714ec57a30e5a6fa592a68e0df (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-24T12:11:24.428 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.185Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-03-24T12:11:24.428 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.185Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-03-24T12:11:24.428 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.185Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-03-24T12:11:24.429 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.185Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-03-24T12:11:24.429 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.186Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-03-24T12:11:24.429 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.186Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-03-24T12:11:24.429 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.186Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-03-24T12:11:24.429 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.186Z caller=node_exporter.go:117 level=info collector=arp 2024-03-24T12:11:24.429 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.186Z caller=node_exporter.go:117 level=info collector=bcache 2024-03-24T12:11:24.429 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.186Z caller=node_exporter.go:117 level=info collector=bonding 2024-03-24T12:11:24.429 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.186Z caller=node_exporter.go:117 level=info collector=btrfs 2024-03-24T12:11:24.429 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.186Z caller=node_exporter.go:117 level=info collector=conntrack 2024-03-24T12:11:24.429 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=cpu 2024-03-24T12:11:24.429 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-03-24T12:11:24.429 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=diskstats 2024-03-24T12:11:24.429 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=dmi 2024-03-24T12:11:24.430 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=edac 2024-03-24T12:11:24.430 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=entropy 2024-03-24T12:11:24.430 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-03-24T12:11:24.430 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=filefd 2024-03-24T12:11:24.430 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=filesystem 2024-03-24T12:11:24.430 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=hwmon 2024-03-24T12:11:24.430 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=infiniband 2024-03-24T12:11:24.430 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=ipvs 2024-03-24T12:11:24.431 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=loadavg 2024-03-24T12:11:24.431 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=mdadm 2024-03-24T12:11:24.431 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=meminfo 2024-03-24T12:11:24.431 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=netclass 2024-03-24T12:11:24.431 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=netdev 2024-03-24T12:11:24.431 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=netstat 2024-03-24T12:11:24.431 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=nfs 2024-03-24T12:11:24.431 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=nfsd 2024-03-24T12:11:24.431 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=nvme 2024-03-24T12:11:24.431 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=os 2024-03-24T12:11:24.431 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-03-24T12:11:24.431 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=pressure 2024-03-24T12:11:24.431 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=rapl 2024-03-24T12:11:24.431 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=schedstat 2024-03-24T12:11:24.431 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=selinux 2024-03-24T12:11:24.432 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=sockstat 2024-03-24T12:11:24.432 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=softnet 2024-03-24T12:11:24.432 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=stat 2024-03-24T12:11:24.432 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=tapestats 2024-03-24T12:11:24.432 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=textfile 2024-03-24T12:11:24.432 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-03-24T12:11:24.432 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=time 2024-03-24T12:11:24.432 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-03-24T12:11:24.432 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=uname 2024-03-24T12:11:24.432 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=vmstat 2024-03-24T12:11:24.432 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=xfs 2024-03-24T12:11:24.432 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=node_exporter.go:117 level=info collector=zfs 2024-03-24T12:11:24.432 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-03-24T12:11:24.432 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b[67004]: ts=2024-03-24T12:11:24.187Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-03-24T12:11:24.432 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 bash[66955]: f2c2d5ff5c48d84f23206dd7156f71519e0a07714ec57a30e5a6fa592a68e0df 2024-03-24T12:11:24.433 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:11:24 smithi090 systemd[1]: Started Ceph node-exporter.b for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:11:24.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-mon[33646]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:24.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:24.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:24.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:24.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:24.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:24.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:24 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:11:24.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:24 smithi006 ceph-mon[30108]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:24.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:24 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:24.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:24 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:24.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:24 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:24.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:24 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:24.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:24 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:24.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:24 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:11:24.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:24 smithi006 ceph-mon[41734]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:24.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:24 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:24.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:24 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:24.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:24 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:24.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:24 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:24.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:24 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:24.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:24 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:11:25.620 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:11:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:11:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:11:26.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:26 smithi090 ceph-mon[33646]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:26.770 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:26 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:26.778 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:26 smithi006 ceph-mon[30108]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:26.779 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:26 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:26.779 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:26 smithi006 ceph-mon[41734]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:26.779 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:26 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:28.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:28 smithi090 ceph-mon[33646]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:28.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:28 smithi006 ceph-mon[30108]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:28.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:28 smithi006 ceph-mon[41734]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:30.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:29 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:30.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:29 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:30.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:29 smithi090 ceph-mon[33646]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:30.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:29 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:30.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:29 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:30.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:29 smithi006 ceph-mon[30108]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:30.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:29 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:30.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:29 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:30.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:29 smithi006 ceph-mon[41734]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:31.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:31 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:31.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:31 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:31.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:31 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:11:31.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:31 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:11:31.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:31 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:31.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:31 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:11:31.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:31 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:31.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:31 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:31.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:31 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:11:31.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:31 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:11:31.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:31 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:31.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:31 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:11:31.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:31 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:31.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:31 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:31.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:31 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:11:31.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:31 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:11:31.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:31 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:31.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:31 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:11:32.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:11:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:11:32.873 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:32 smithi006 ceph-mon[30108]: Reconfiguring alertmanager.a (dependencies changed)... 2024-03-24T12:11:32.874 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:32 smithi006 ceph-mon[30108]: Reconfiguring daemon alertmanager.a on smithi006 2024-03-24T12:11:32.874 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:32 smithi006 ceph-mon[30108]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:32.874 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:32 smithi006 ceph-mon[41734]: Reconfiguring alertmanager.a (dependencies changed)... 2024-03-24T12:11:32.874 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:32 smithi006 ceph-mon[41734]: Reconfiguring daemon alertmanager.a on smithi006 2024-03-24T12:11:32.874 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:32 smithi006 ceph-mon[41734]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:32.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:32 smithi090 ceph-mon[33646]: Reconfiguring alertmanager.a (dependencies changed)... 2024-03-24T12:11:32.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:32 smithi090 ceph-mon[33646]: Reconfiguring daemon alertmanager.a on smithi006 2024-03-24T12:11:32.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:32 smithi090 ceph-mon[33646]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:33.796 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:33 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:11:33.796 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:33 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:11:33.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:33 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:11:34.661 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:34 smithi006 ceph-mon[30108]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:34.661 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:34 smithi006 ceph-mon[41734]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:34.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:34 smithi090 ceph-mon[33646]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:35.261 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:35 smithi006 systemd[1]: Stopping Ceph alertmanager.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:11:35.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:11:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:11:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:11:35.711 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a[79411]: ts=2024-03-24T12:11:35.477Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-03-24T12:11:35.711 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:35 smithi006 podman[80810]: 2024-03-24 12:11:35.498659984 +0000 UTC m=+0.133678276 container died bba13843ec61747d522d369757ecee450b507e8be0fd8406d985ee0522ede4ae (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T12:11:36.100 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:35 smithi006 podman[80810]: 2024-03-24 12:11:35.917071578 +0000 UTC m=+0.552089885 container cleanup bba13843ec61747d522d369757ecee450b507e8be0fd8406d985ee0522ede4ae (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T12:11:36.100 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:35 smithi006 bash[80810]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a 2024-03-24T12:11:36.460 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:36 smithi006 podman[80825]: 2024-03-24 12:11:36.101585438 +0000 UTC m=+0.606301966 container remove bba13843ec61747d522d369757ecee450b507e8be0fd8406d985ee0522ede4ae (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T12:11:36.461 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:36 smithi006 podman[80825]: 2024-03-24 12:11:36.20221502 +0000 UTC m=+0.706931556 volume remove 37a356bf24e4ca5a3d38cf41b3c6e6a246e203d842760e35179ecb5d65e2e0ea 2024-03-24T12:11:36.877 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:36 smithi006 ceph-mon[30108]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:11:36.878 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:36 smithi006 ceph-mon[41734]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:11:36.878 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:36 smithi006 systemd[1]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-03-24T12:11:36.878 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:36 smithi006 systemd[1]: Stopped Ceph alertmanager.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:11:36.878 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:36 smithi006 systemd[1]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@alertmanager.a.service: Consumed 1.026s CPU time. 2024-03-24T12:11:36.878 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:36 smithi006 systemd[1]: Starting Ceph alertmanager.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:11:36.878 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:36 smithi006 podman[80921]: 2024-03-24 12:11:36.688199359 +0000 UTC m=+0.025971725 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-03-24T12:11:36.878 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:36 smithi006 podman[80921]: 2024-03-24 12:11:36.786596443 +0000 UTC m=+0.124368791 volume create 17adfea2088e0620e0aff02152fc0bdd6823cbd15a3533fc30e58c510e31366c 2024-03-24T12:11:36.878 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:36 smithi006 podman[80921]: 2024-03-24 12:11:36.878683459 +0000 UTC m=+0.216455852 container create 52006839997a02f5b023ac7c4a78a7cda3c324576a621d404782cfbcbdc3ec4e (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T12:11:36.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:36 smithi090 ceph-mon[33646]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:11:37.197 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:37 smithi006 podman[80921]: 2024-03-24 12:11:37.198573732 +0000 UTC m=+0.536346100 container init 52006839997a02f5b023ac7c4a78a7cda3c324576a621d404782cfbcbdc3ec4e (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T12:11:37.461 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:37 smithi006 podman[80921]: 2024-03-24 12:11:37.206835774 +0000 UTC m=+0.544608125 container start 52006839997a02f5b023ac7c4a78a7cda3c324576a621d404782cfbcbdc3ec4e (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T12:11:37.461 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:37 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a[80935]: ts=2024-03-24T12:11:37.221Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-03-24T12:11:37.461 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:37 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a[80935]: ts=2024-03-24T12:11:37.221Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-03-24T12:11:37.461 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:37 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a[80935]: ts=2024-03-24T12:11:37.222Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.6 port=9094 2024-03-24T12:11:37.461 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:37 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a[80935]: ts=2024-03-24T12:11:37.223Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-03-24T12:11:37.461 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:37 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a[80935]: ts=2024-03-24T12:11:37.256Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-24T12:11:37.462 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:37 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a[80935]: ts=2024-03-24T12:11:37.257Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-24T12:11:37.462 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:37 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a[80935]: ts=2024-03-24T12:11:37.260Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-03-24T12:11:37.462 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:37 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a[80935]: ts=2024-03-24T12:11:37.261Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-03-24T12:11:37.462 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:37 smithi006 bash[80921]: 52006839997a02f5b023ac7c4a78a7cda3c324576a621d404782cfbcbdc3ec4e 2024-03-24T12:11:37.462 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:37 smithi006 systemd[1]: Started Ceph alertmanager.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:11:38.543 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:38 smithi090 ceph-mon[33646]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:38.544 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:38 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:38.544 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:38 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:38.544 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:38 smithi090 ceph-mon[33646]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-24T12:11:38.638 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:38 smithi006 ceph-mon[30108]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:38.638 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:38 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:38.638 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:38 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:38.639 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:38 smithi006 ceph-mon[30108]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-24T12:11:38.639 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:38 smithi006 ceph-mon[41734]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:38.639 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:38 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:38.639 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:38 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:38.639 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:38 smithi006 ceph-mon[41734]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-24T12:11:39.486 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:39 smithi090 ceph-mon[33646]: Reconfiguring daemon prometheus.a on smithi090 2024-03-24T12:11:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:39 smithi006 ceph-mon[30108]: Reconfiguring daemon prometheus.a on smithi090 2024-03-24T12:11:39.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:39 smithi006 ceph-mon[41734]: Reconfiguring daemon prometheus.a on smithi090 2024-03-24T12:11:39.711 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:39 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a[80935]: ts=2024-03-24T12:11:39.224Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000413217s 2024-03-24T12:11:40.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:40 smithi090 ceph-mon[33646]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:11:40.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:40 smithi006 ceph-mon[30108]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:11:40.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:40 smithi006 ceph-mon[41734]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:11:41.178 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:40 smithi090 systemd[1]: Stopping Ceph prometheus.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:11:41.512 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:41 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:11:41.230Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-03-24T12:11:41.512 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:41 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:11:41.230Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-03-24T12:11:41.513 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:41 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:11:41.230Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-03-24T12:11:41.513 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:41 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:11:41.230Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-03-24T12:11:41.513 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:41 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:11:41.230Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-03-24T12:11:41.513 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:41 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:11:41.230Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-03-24T12:11:41.513 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:41 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:11:41.230Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-03-24T12:11:41.514 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:41 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:11:41.230Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-03-24T12:11:41.514 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:41 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:11:41.231Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-03-24T12:11:41.514 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:41 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:11:41.231Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-03-24T12:11:41.514 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:41 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:11:41.231Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-03-24T12:11:41.514 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:41 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[64683]: ts=2024-03-24T12:11:41.231Z caller=main.go:1147 level=info msg="See you next time!" 2024-03-24T12:11:41.515 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:41 smithi090 podman[68044]: 2024-03-24 12:11:41.241262222 +0000 UTC m=+0.124582073 container died 3bb67e0745abf3e70a7aca2c44ea0b0e3f493aee962755df2a0db4cc2e91918b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T12:11:41.854 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:41 smithi090 podman[68044]: 2024-03-24 12:11:41.612616607 +0000 UTC m=+0.495936459 container cleanup 3bb67e0745abf3e70a7aca2c44ea0b0e3f493aee962755df2a0db4cc2e91918b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T12:11:41.854 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:41 smithi090 bash[68044]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a 2024-03-24T12:11:42.178 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:11:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:11:42.179 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:41 smithi090 podman[68063]: 2024-03-24 12:11:41.855772207 +0000 UTC m=+0.614694766 container remove 3bb67e0745abf3e70a7aca2c44ea0b0e3f493aee962755df2a0db4cc2e91918b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T12:11:42.179 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:41 smithi090 systemd[1]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@prometheus.a.service: Deactivated successfully. 2024-03-24T12:11:42.179 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:41 smithi090 systemd[1]: Stopped Ceph prometheus.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:11:42.179 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:41 smithi090 systemd[1]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@prometheus.a.service: Consumed 1.155s CPU time. 2024-03-24T12:11:42.179 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:41 smithi090 systemd[1]: Starting Ceph prometheus.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:11:42.461 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:42 smithi090 ceph-mon[33646]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:11:42.461 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 podman[68162]: 2024-03-24 12:11:42.098488974 +0000 UTC m=+0.019307147 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-03-24T12:11:42.461 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 podman[68162]: 2024-03-24 12:11:42.223875557 +0000 UTC m=+0.144693750 container create 72dfed8906bfef48481814a6886e4434cb6cb8c00aaf5d4c6f64d039fdc5302d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T12:11:42.461 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 podman[68162]: 2024-03-24 12:11:42.4344256 +0000 UTC m=+0.355243758 container init 72dfed8906bfef48481814a6886e4434cb6cb8c00aaf5d4c6f64d039fdc5302d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T12:11:42.461 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 podman[68162]: 2024-03-24 12:11:42.439614982 +0000 UTC m=+0.360433133 container start 72dfed8906bfef48481814a6886e4434cb6cb8c00aaf5d4c6f64d039fdc5302d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T12:11:42.637 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:42 smithi006 ceph-mon[30108]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:11:42.638 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:42 smithi006 ceph-mon[41734]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:11:42.928 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:11:42.462Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-03-24T12:11:42.929 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:11:42.462Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-03-24T12:11:42.929 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:11:42.462Z caller=main.go:570 level=info host_details="(Linux 5.14.0-430.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024 x86_64 smithi090 (none))" 2024-03-24T12:11:42.929 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:11:42.462Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-03-24T12:11:42.929 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:11:42.462Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-03-24T12:11:42.929 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:11:42.464Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-03-24T12:11:42.929 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:11:42.464Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-03-24T12:11:42.929 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:11:42.465Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-03-24T12:11:42.929 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:11:42.465Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-03-24T12:11:42.929 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:11:42.467Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-03-24T12:11:42.929 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:11:42.467Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.432µs 2024-03-24T12:11:42.929 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:11:42.467Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-03-24T12:11:42.929 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:11:42.468Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-03-24T12:11:42.930 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:11:42.468Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-03-24T12:11:42.930 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:11:42.468Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=18.468µs wal_replay_duration=694.967µs wbl_replay_duration=101ns total_replay_duration=730.918µs 2024-03-24T12:11:42.930 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:11:42.469Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-03-24T12:11:42.930 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:11:42.469Z caller=main.go:1029 level=info msg="TSDB started" 2024-03-24T12:11:42.930 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:11:42.469Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-03-24T12:11:42.930 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:11:42.486Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=17.374571ms db_storage=743ns remote_storage=1.371µs web_handler=308ns query_engine=493ns scrape=369.373µs scrape_sd=72.713µs notify=9.808µs notify_sd=13.238µs rules=16.654779ms tracing=6.025µs 2024-03-24T12:11:42.930 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:11:42.486Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-03-24T12:11:42.930 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:11:42.486Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-03-24T12:11:42.930 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 bash[68162]: 72dfed8906bfef48481814a6886e4434cb6cb8c00aaf5d4c6f64d039fdc5302d 2024-03-24T12:11:42.930 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:11:42 smithi090 systemd[1]: Started Ceph prometheus.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:11:42.961 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:11:42 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:11:42] ENGINE Bus STOPPING 2024-03-24T12:11:42.961 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:11:42 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:11:42] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-24T12:11:42.961 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:11:42 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:11:42] ENGINE Bus STOPPED 2024-03-24T12:11:42.961 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:11:42 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:11:42] ENGINE Bus STARTING 2024-03-24T12:11:42.961 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:11:42 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:11:42] ENGINE Serving on http://:::9283 2024-03-24T12:11:42.962 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:11:42 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:11:42] ENGINE Bus STARTED 2024-03-24T12:11:42.962 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:11:42 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:11:42] ENGINE Bus STOPPING 2024-03-24T12:11:42.962 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:11:42 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:11:42] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-24T12:11:42.962 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:11:42 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:11:42] ENGINE Bus STOPPED 2024-03-24T12:11:42.962 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:11:42 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:11:42] ENGINE Bus STARTING 2024-03-24T12:11:42.962 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:11:42 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:11:42] ENGINE Serving on http://:::9283 2024-03-24T12:11:42.962 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:11:42 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:11:42] ENGINE Bus STARTED 2024-03-24T12:11:42.962 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:11:42 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:11:42] ENGINE Bus STOPPING 2024-03-24T12:11:42.962 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:11:42 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:11:42] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-24T12:11:42.962 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:11:42 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:11:42] ENGINE Bus STOPPED 2024-03-24T12:11:42.962 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:11:42 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:11:42] ENGINE Bus STARTING 2024-03-24T12:11:43.461 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:11:42 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:11:42] ENGINE Serving on http://:::9283 2024-03-24T12:11:43.461 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:11:42 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:11:42] ENGINE Bus STARTED 2024-03-24T12:11:43.875 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:11:43.876 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:43.876 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:43.876 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T12:11:43.876 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi006.front.sepia.ceph.com:9093"}]: dispatch 2024-03-24T12:11:43.876 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:43.876 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T12:11:43.876 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi090.front.sepia.ceph.com:9095"}]: dispatch 2024-03-24T12:11:43.876 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:43.876 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T12:11:43.876 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi090.front.sepia.ceph.com:3000"}]: dispatch 2024-03-24T12:11:43.876 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:43.876 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:11:43.877 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:11:43.877 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:43.877 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:43.877 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T12:11:43.877 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi006.front.sepia.ceph.com:9093"}]: dispatch 2024-03-24T12:11:43.877 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:43.877 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T12:11:43.877 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi090.front.sepia.ceph.com:9095"}]: dispatch 2024-03-24T12:11:43.877 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:43.877 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T12:11:43.877 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi090.front.sepia.ceph.com:3000"}]: dispatch 2024-03-24T12:11:43.878 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:43.878 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:43 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:11:43.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:43 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:11:43.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:43 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:43.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:43 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:43.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:43 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T12:11:43.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:43 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi006.front.sepia.ceph.com:9093"}]: dispatch 2024-03-24T12:11:43.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:43 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:43.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:43 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T12:11:43.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:43 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi090.front.sepia.ceph.com:9095"}]: dispatch 2024-03-24T12:11:43.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:43 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:43.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:43 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T12:11:43.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:43 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi090.front.sepia.ceph.com:3000"}]: dispatch 2024-03-24T12:11:43.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:43 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:43.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:43 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:11:44.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:44 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T12:11:44.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:44 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi006.front.sepia.ceph.com:9093"}]: dispatch 2024-03-24T12:11:44.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:44 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T12:11:44.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:44 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi090.front.sepia.ceph.com:9095"}]: dispatch 2024-03-24T12:11:44.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:44 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T12:11:44.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:44 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi090.front.sepia.ceph.com:3000"}]: dispatch 2024-03-24T12:11:44.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:44 smithi090 ceph-mon[33646]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:44.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:44 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T12:11:44.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:44 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi006.front.sepia.ceph.com:9093"}]: dispatch 2024-03-24T12:11:44.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:44 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T12:11:44.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:44 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi090.front.sepia.ceph.com:9095"}]: dispatch 2024-03-24T12:11:44.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:44 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T12:11:44.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:44 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi090.front.sepia.ceph.com:3000"}]: dispatch 2024-03-24T12:11:44.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:44 smithi006 ceph-mon[30108]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:44.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:44 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-24T12:11:44.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:44 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi006.front.sepia.ceph.com:9093"}]: dispatch 2024-03-24T12:11:44.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:44 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-24T12:11:44.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:44 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi090.front.sepia.ceph.com:9095"}]: dispatch 2024-03-24T12:11:44.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:44 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-24T12:11:44.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:44 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi090.front.sepia.ceph.com:3000"}]: dispatch 2024-03-24T12:11:44.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:44 smithi006 ceph-mon[41734]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:46.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:46 smithi090 ceph-mon[33646]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:46.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:46 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:11:46.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:46 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:46.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:46 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:46.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:46 smithi006 ceph-mon[30108]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:46.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:46 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:11:46.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:46 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:46.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:46 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:46.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:46 smithi006 ceph-mon[41734]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:46.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:46 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:11:46.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:46 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:46.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:46 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:47.710 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:11:47 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a[80935]: ts=2024-03-24T12:11:47.227Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003578028s 2024-03-24T12:11:48.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:48 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:48.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:48 smithi090 ceph-mon[33646]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:48.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:48 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:48.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:48 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:11:48.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:48 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:11:48.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:48 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:48.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:48 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:48.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:48 smithi006 ceph-mon[30108]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:48.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:48 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:48.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:48 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:11:48.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:48 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:11:48.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:48 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:48.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:48 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:48.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:48 smithi006 ceph-mon[41734]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:48.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:48 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:48.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:48 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:11:48.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:48 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:11:48.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:48 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:11:50.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:50 smithi090 ceph-mon[33646]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:50.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:50 smithi006 ceph-mon[30108]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:50.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:50 smithi006 ceph-mon[41734]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:52.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:52 smithi090 ceph-mon[33646]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:52.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:11:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:11:52.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:52 smithi006 ceph-mon[30108]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:52.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:52 smithi006 ceph-mon[41734]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:53.432 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:53 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:11:53.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:53 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:11:53.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:53 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:11:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:54 smithi090 ceph-mon[33646]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:54.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:54 smithi006 ceph-mon[30108]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:54.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:54 smithi006 ceph-mon[41734]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:55.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:11:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:11:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:11:56.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:56 smithi090 ceph-mon[33646]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:56.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:56 smithi006 ceph-mon[30108]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:56.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:56 smithi006 ceph-mon[41734]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:11:58.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:11:58 smithi090 ceph-mon[33646]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:58.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:11:58 smithi006 ceph-mon[30108]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:58.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:11:58 smithi006 ceph-mon[41734]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:11:59.557 INFO:tasks.workunit.client.0.smithi006.stderr:Updating files: 99% (12329/12399) Updating files: 100% (12399/12399) Updating files: 100% (12399/12399), done. 2024-03-24T12:12:00.187 INFO:tasks.workunit.client.0.smithi006.stderr:Note: switching to 'f73f1dfd96849300c4322bd7259a3ab37f94c200'. 2024-03-24T12:12:00.189 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-03-24T12:12:00.189 INFO:tasks.workunit.client.0.smithi006.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-24T12:12:00.189 INFO:tasks.workunit.client.0.smithi006.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-24T12:12:00.189 INFO:tasks.workunit.client.0.smithi006.stderr:state without impacting any branches by switching back to a branch. 2024-03-24T12:12:00.189 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-03-24T12:12:00.190 INFO:tasks.workunit.client.0.smithi006.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-24T12:12:00.190 INFO:tasks.workunit.client.0.smithi006.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-24T12:12:00.190 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-03-24T12:12:00.190 INFO:tasks.workunit.client.0.smithi006.stderr: git switch -c 2024-03-24T12:12:00.190 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-03-24T12:12:00.190 INFO:tasks.workunit.client.0.smithi006.stderr:Or undo this operation with: 2024-03-24T12:12:00.190 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-03-24T12:12:00.190 INFO:tasks.workunit.client.0.smithi006.stderr: git switch - 2024-03-24T12:12:00.190 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-03-24T12:12:00.190 INFO:tasks.workunit.client.0.smithi006.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-24T12:12:00.190 INFO:tasks.workunit.client.0.smithi006.stderr: 2024-03-24T12:12:00.190 INFO:tasks.workunit.client.0.smithi006.stderr:HEAD is now at f73f1dfd968 Merge pull request #56388 from rhcs-dashboard/wip-64960-squid 2024-03-24T12:12:00.194 DEBUG:teuthology.orchestra.run.smithi006:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-24T12:12:00.216 INFO:tasks.workunit.client.0.smithi006.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-24T12:12:00.218 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-24T12:12:00.218 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-24T12:12:00.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:00 smithi090 ceph-mon[33646]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:12:00.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:00 smithi006 ceph-mon[30108]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:12:00.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:00 smithi006 ceph-mon[41734]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:12:00.996 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-24T12:12:01.065 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-24T12:12:01.104 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-24T12:12:01.106 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-24T12:12:01.106 INFO:tasks.workunit.client.0.smithi006.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-24T12:12:01.148 INFO:tasks.workunit.client.0.smithi006.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-24T12:12:01.152 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2024-03-24T12:12:01.152 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-24T12:12:01.207 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-03-24T12:12:01.208 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-03-24T12:12:01.208 DEBUG:teuthology.orchestra.run.smithi006:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f73f1dfd96849300c4322bd7259a3ab37f94c200 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-03-24T12:12:01.325 INFO:tasks.workunit.client.0.smithi006.stderr:+ parallel=1 2024-03-24T12:12:01.325 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' '' = --serial ']' 2024-03-24T12:12:01.325 INFO:tasks.workunit.client.0.smithi006.stderr:+ crimson=0 2024-03-24T12:12:01.325 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' '' = --crimson ']' 2024-03-24T12:12:01.325 INFO:tasks.workunit.client.0.smithi006.stderr:+ color= 2024-03-24T12:12:01.325 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -t 1 ']' 2024-03-24T12:12:01.325 INFO:tasks.workunit.client.0.smithi006.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-03-24T12:12:01.326 INFO:tasks.workunit.client.0.smithi006.stderr:+ GTEST_OUTPUT_DIR=/home/ubuntu/cephtest/archive/unit_test_xml_report 2024-03-24T12:12:01.326 INFO:tasks.workunit.client.0.smithi006.stderr:+ mkdir -p /home/ubuntu/cephtest/archive/unit_test_xml_report 2024-03-24T12:12:01.326 INFO:tasks.workunit.client.0.smithi006.stderr:+ declare -A pids 2024-03-24T12:12:01.326 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.326 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.326 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_aio 2024-03-24T12:12:01.326 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_aio' 2024-03-24T12:12:01.327 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_aio 2024-03-24T12:12:01.327 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.327 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_aio 2024-03-24T12:12:01.327 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81708 2024-03-24T12:12:01.327 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_aio on pid 81708' 2024-03-24T12:12:01.327 INFO:tasks.workunit.client.0.smithi006.stdout:test api_aio on pid 81708 2024-03-24T12:12:01.330 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=81708 2024-03-24T12:12:01.330 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.330 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.330 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-03-24T12:12:01.330 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_aio_pp 2024-03-24T12:12:01.330 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_aio_pp' 2024-03-24T12:12:01.330 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_aio_pp 2024-03-24T12:12:01.331 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.331 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.331 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.331 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.331 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.331 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.331 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.331 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.331 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.331 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.331 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.331 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.331 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.331 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.332 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.332 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.332 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.332 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.332 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.332 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.332 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.332 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.332 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.332 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.332 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.332 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.332 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_aio_pp 2024-03-24T12:12:01.332 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81715 2024-03-24T12:12:01.333 INFO:tasks.workunit.client.0.smithi006.stdout:test api_aio_pp on pid 81715 2024-03-24T12:12:01.337 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_aio_pp on pid 81715' 2024-03-24T12:12:01.337 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=81715 2024-03-24T12:12:01.337 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.337 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.338 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-03-24T12:12:01.338 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_io 2024-03-24T12:12:01.338 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_io' 2024-03-24T12:12:01.338 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_io 2024-03-24T12:12:01.338 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.338 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.338 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.338 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.338 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.338 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.338 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.338 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.338 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.339 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.339 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.339 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.339 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.339 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.339 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.339 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.339 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.339 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.339 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.339 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.339 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.339 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.339 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.339 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.340 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.340 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.340 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.340 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.340 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.340 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.340 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.340 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.340 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.340 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.340 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.340 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.340 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.340 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.340 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.340 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.341 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.341 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.341 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.341 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.341 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.341 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.341 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_io 2024-03-24T12:12:01.341 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81724 2024-03-24T12:12:01.341 INFO:tasks.workunit.client.0.smithi006.stdout:test api_io on pid 81724 2024-03-24T12:12:01.345 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_io on pid 81724' 2024-03-24T12:12:01.345 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=81724 2024-03-24T12:12:01.345 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.346 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-03-24T12:12:01.346 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.346 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_io_pp 2024-03-24T12:12:01.346 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_io_pp' 2024-03-24T12:12:01.346 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_io_pp 2024-03-24T12:12:01.346 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.346 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.346 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.346 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.346 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.346 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.346 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.346 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.346 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.347 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.347 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.347 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.347 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.347 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.347 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.347 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.347 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.347 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.347 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.347 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.347 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.347 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.347 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.347 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.348 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.348 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.348 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.348 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.348 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.348 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.348 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.348 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.348 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.348 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.348 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.348 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.348 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.348 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.348 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.349 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.349 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.349 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.349 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.349 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.349 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.349 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.349 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.349 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.349 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.349 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.349 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.349 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.349 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.349 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.350 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.350 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_io_pp 2024-03-24T12:12:01.350 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81736 2024-03-24T12:12:01.350 INFO:tasks.workunit.client.0.smithi006.stdout:test api_io_pp on pid 81736 2024-03-24T12:12:01.354 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_io_pp on pid 81736' 2024-03-24T12:12:01.355 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=81736 2024-03-24T12:12:01.355 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-03-24T12:12:01.355 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.355 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.355 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_asio 2024-03-24T12:12:01.355 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_asio' 2024-03-24T12:12:01.355 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_asio 2024-03-24T12:12:01.355 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.355 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.355 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.355 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.355 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.355 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.356 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.356 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.356 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.356 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.356 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.356 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.356 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.356 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.356 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.356 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.356 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.356 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.356 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.356 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.356 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.356 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.357 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.357 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.357 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.357 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.357 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.357 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.357 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.357 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.357 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.357 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.357 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.357 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.357 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.357 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.357 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.358 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.358 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.358 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.358 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.358 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.358 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.358 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.358 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.358 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.358 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.358 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.358 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.358 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.358 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.359 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.359 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.359 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.359 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.359 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.359 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.359 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.359 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.359 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.359 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.359 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.359 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.359 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.359 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.359 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.359 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.360 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.360 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.360 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.360 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.360 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.360 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.360 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_asio 2024-03-24T12:12:01.360 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.360 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.360 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81749 2024-03-24T12:12:01.360 INFO:tasks.workunit.client.0.smithi006.stdout:test api_asio on pid 81749 2024-03-24T12:12:01.366 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_asio on pid 81749' 2024-03-24T12:12:01.366 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=81749 2024-03-24T12:12:01.366 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.366 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.366 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.366 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.366 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.366 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-03-24T12:12:01.366 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_list 2024-03-24T12:12:01.366 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.367 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_list' 2024-03-24T12:12:01.367 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_list 2024-03-24T12:12:01.367 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.367 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.367 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.367 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.367 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.367 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.367 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.367 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.367 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.367 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.367 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.367 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.367 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.368 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.368 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.368 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.368 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.368 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.368 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.368 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.368 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.368 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.368 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.368 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.368 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.368 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.368 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.369 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.369 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.369 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.369 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.369 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.369 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.369 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.369 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.369 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.369 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.369 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.369 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.369 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.369 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.369 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.370 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.370 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.370 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.370 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.370 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.370 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.370 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.370 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.370 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.370 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.370 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.371 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.371 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.371 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.371 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.371 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.371 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.371 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.371 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.371 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.371 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.371 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.371 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.371 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.371 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.372 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.372 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.372 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.372 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.372 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.372 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.372 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.372 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.372 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.372 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.372 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.372 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.372 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.372 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.373 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.373 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.373 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.373 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.373 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.373 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.373 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.373 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.373 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.373 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.373 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.373 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.373 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.373 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/81708/exe 2024-03-24T12:12:01.374 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_list 2024-03-24T12:12:01.374 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81767 2024-03-24T12:12:01.374 INFO:tasks.workunit.client.0.smithi006.stdout:test api_list on pid 81767 2024-03-24T12:12:01.381 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_list on pid 81767' 2024-03-24T12:12:01.381 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=81767 2024-03-24T12:12:01.381 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.381 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.381 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-03-24T12:12:01.381 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.381 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.381 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.381 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.382 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_lock 2024-03-24T12:12:01.382 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.382 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.382 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_lock' 2024-03-24T12:12:01.382 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.382 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_lock 2024-03-24T12:12:01.382 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.382 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.382 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.382 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.382 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.382 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.382 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.382 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.383 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.383 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.383 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.383 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.383 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.383 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.383 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.383 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.383 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.383 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.383 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.383 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.383 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.383 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.383 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.384 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.384 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.384 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.384 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.384 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.384 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.384 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.384 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.384 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.384 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.384 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.384 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.384 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.384 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.385 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.385 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.385 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.385 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.385 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.385 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.385 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2024-03-24T12:12:01.385 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.385 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_aio.log 2024-03-24T12:12:01.385 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.385 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.385 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.385 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.385 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.385 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.386 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.386 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.386 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.386 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.386 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.386 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.386 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.386 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.386 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.386 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.386 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.386 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.386 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.386 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.387 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.387 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.387 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.387 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.387 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.387 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.387 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.387 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.387 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.387 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.387 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.387 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.387 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.387 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.387 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.387 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.388 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.388 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.388 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.388 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.388 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.388 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.388 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.388 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.388 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.388 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.388 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.388 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.388 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.388 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.388 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.389 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.389 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.389 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.389 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.389 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.389 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.389 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.389 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.389 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_aio: /' 2024-03-24T12:12:01.389 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.389 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/81715/exe 2024-03-24T12:12:01.389 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.389 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.389 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.389 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.390 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.390 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.390 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.390 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_lock 2024-03-24T12:12:01.390 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81789 2024-03-24T12:12:01.390 INFO:tasks.workunit.client.0.smithi006.stdout:test api_lock on pid 81789 2024-03-24T12:12:01.396 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_lock on pid 81789' 2024-03-24T12:12:01.396 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=81789 2024-03-24T12:12:01.396 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.396 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.396 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.396 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.396 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.397 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.397 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.397 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_lock_pp 2024-03-24T12:12:01.397 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.397 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.397 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.397 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.397 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.397 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.397 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.397 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_lock_pp' 2024-03-24T12:12:01.397 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.397 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.398 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.398 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.398 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.398 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.398 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.398 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.398 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.398 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.398 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.398 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.398 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.398 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.398 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.398 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.398 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-03-24T12:12:01.399 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.399 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.399 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.399 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.399 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_lock_pp 2024-03-24T12:12:01.399 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.399 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.399 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.399 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.399 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.399 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.399 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.399 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.399 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.399 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.400 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.400 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.400 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.400 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.400 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.400 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_aio_pp: /' 2024-03-24T12:12:01.400 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.400 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.400 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.400 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.400 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.400 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.400 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.400 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.400 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.401 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.401 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.401 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2024-03-24T12:12:01.401 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.401 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.401 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.401 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.401 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.401 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.401 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.401 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.401 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.401 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.401 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.401 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.402 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.402 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.402 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.402 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.402 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.402 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.402 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.402 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.402 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.402 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.402 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.402 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.402 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-03-24T12:12:01.402 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.402 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.402 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.403 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/81724/exe 2024-03-24T12:12:01.403 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.403 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.403 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.403 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.403 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.403 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.403 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.403 INFO:tasks.workunit.client.0.smithi006.stdout:test api_lock_pp on pid 81807 2024-03-24T12:12:01.408 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.408 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.408 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.408 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_lock_pp 2024-03-24T12:12:01.408 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.408 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.408 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.408 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.409 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.409 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.409 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.409 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.409 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.409 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81807 2024-03-24T12:12:01.409 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_lock_pp on pid 81807' 2024-03-24T12:12:01.409 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=81807 2024-03-24T12:12:01.409 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.409 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.409 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.409 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-03-24T12:12:01.409 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_misc 2024-03-24T12:12:01.409 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.410 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.410 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.410 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.410 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.410 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.410 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.410 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_misc' 2024-03-24T12:12:01.410 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.410 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.410 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.410 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.410 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.410 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.410 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.411 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.411 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.411 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.411 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.411 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.411 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.411 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.411 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.411 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.411 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.411 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.411 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_io.log 2024-03-24T12:12:01.411 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.411 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.411 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.412 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.412 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.412 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.412 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.412 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.412 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.412 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.412 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.412 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.412 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.412 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.412 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.412 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.412 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.413 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.413 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.413 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.413 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.413 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.413 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.413 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.413 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_io: /' 2024-03-24T12:12:01.413 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.413 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.413 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.413 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.413 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.413 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.413 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2024-03-24T12:12:01.414 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.414 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.414 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.414 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.414 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.414 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.414 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.414 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.414 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.414 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.414 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.414 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.414 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.414 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.414 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.415 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.415 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.415 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.415 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.415 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.415 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.415 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.415 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.415 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.416 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.416 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.416 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.416 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.416 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.416 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.416 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.416 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.416 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.416 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.416 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.416 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.416 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.416 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.416 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.417 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.417 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.417 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.417 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.417 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.417 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.417 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.417 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.417 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.417 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.417 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/81736/exe 2024-03-24T12:12:01.417 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.417 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.417 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.417 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.418 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.418 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.418 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.418 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.418 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.418 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.418 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.418 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.418 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.418 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.418 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.418 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.418 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.418 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.419 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.419 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.419 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.419 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.419 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.419 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.419 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.419 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.419 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.419 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.419 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.419 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.419 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_misc 2024-03-24T12:12:01.419 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.419 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.420 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.420 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.420 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.420 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.420 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.420 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.420 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.420 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.420 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.420 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.420 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.420 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.420 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.420 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.420 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.421 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.421 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.421 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.421 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.421 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.421 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.421 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.421 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.421 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.421 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.421 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.421 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.421 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.421 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.421 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.422 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_io_pp: /' 2024-03-24T12:12:01.422 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/81749/exe 2024-03-24T12:12:01.422 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-03-24T12:12:01.422 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_misc 2024-03-24T12:12:01.422 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81836 2024-03-24T12:12:01.422 INFO:tasks.workunit.client.0.smithi006.stdout:test api_misc on pid 81836 2024-03-24T12:12:01.429 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_misc on pid 81836' 2024-03-24T12:12:01.429 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=81836 2024-03-24T12:12:01.429 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.429 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.429 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2024-03-24T12:12:01.429 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.429 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.429 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.429 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.430 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.430 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.430 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.430 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.430 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.430 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.430 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.430 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.430 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.430 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.430 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.430 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.431 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.431 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.431 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-03-24T12:12:01.431 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.431 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.431 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.431 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.431 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.431 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.431 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.431 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.431 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.431 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.431 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.432 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.432 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.432 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.432 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.432 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.432 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.432 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.432 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.432 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.432 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.432 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.432 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.432 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.432 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.433 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.433 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.433 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_asio.log 2024-03-24T12:12:01.433 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.433 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.433 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.433 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.433 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.433 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.433 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.433 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.433 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2024-03-24T12:12:01.433 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_misc_pp 2024-03-24T12:12:01.433 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.434 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.434 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.434 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.434 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.434 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.434 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.434 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.434 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.434 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.434 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.434 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.434 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.434 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.434 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.434 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_misc_pp' 2024-03-24T12:12:01.435 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.435 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.435 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.435 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.435 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.435 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.435 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.435 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.435 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.435 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.435 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.435 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.435 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.435 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.435 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.436 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.436 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.436 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.436 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.436 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.436 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.436 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.436 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.436 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_misc_pp 2024-03-24T12:12:01.436 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.436 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.436 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_asio: /' 2024-03-24T12:12:01.436 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.436 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.436 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.437 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.437 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.437 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.437 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.437 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.437 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.437 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.437 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.437 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.437 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.437 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.437 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.437 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.437 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.437 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.438 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.438 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.438 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.438 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.438 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.438 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.438 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.438 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.438 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.438 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.438 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.438 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.438 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.438 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.438 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.439 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.439 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.439 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.439 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.439 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.439 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.439 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.439 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.439 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.439 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.439 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.439 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.439 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.439 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.440 INFO:tasks.workunit.client.0.smithi006.stdout:test api_misc_pp on pid 81861 2024-03-24T12:12:01.441 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_misc_pp 2024-03-24T12:12:01.441 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/81767/exe 2024-03-24T12:12:01.442 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81861 2024-03-24T12:12:01.442 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_misc_pp on pid 81861' 2024-03-24T12:12:01.442 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=81861 2024-03-24T12:12:01.442 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.442 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.442 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_tier_pp 2024-03-24T12:12:01.442 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.442 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_tier_pp' 2024-03-24T12:12:01.442 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.442 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.442 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.442 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.442 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.442 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.443 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.443 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.443 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.443 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.443 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.443 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.443 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.443 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.443 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.443 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.443 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.443 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-03-24T12:12:01.443 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.443 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.443 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.444 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.444 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.444 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.444 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.444 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.444 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_tier_pp 2024-03-24T12:12:01.444 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.444 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.444 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.444 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.444 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.444 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.444 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.444 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.444 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.445 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.445 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2024-03-24T12:12:01.445 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.445 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.445 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.445 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_list: /' 2024-03-24T12:12:01.445 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.445 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.445 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.445 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.445 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.445 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.446 INFO:tasks.workunit.client.0.smithi006.stdout:test api_tier_pp on pid 81873 2024-03-24T12:12:01.449 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_tier_pp 2024-03-24T12:12:01.449 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81873 2024-03-24T12:12:01.449 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_tier_pp on pid 81873' 2024-03-24T12:12:01.449 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=81873 2024-03-24T12:12:01.450 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.450 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.450 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_list.log 2024-03-24T12:12:01.450 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.450 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.450 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.450 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.450 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.450 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.450 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.450 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.450 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.450 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.450 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.451 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.451 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.451 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.451 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.451 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.451 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.451 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.451 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.451 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.451 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.451 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.451 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.451 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.451 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_pool 2024-03-24T12:12:01.452 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_pool' 2024-03-24T12:12:01.452 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.452 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_pool 2024-03-24T12:12:01.452 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.452 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.452 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.452 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.452 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.452 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.452 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.452 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.452 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.452 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.452 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.452 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.453 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.453 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.453 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.453 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.453 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.453 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.453 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.453 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.453 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.453 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.453 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.453 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.453 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.453 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.453 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.454 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-03-24T12:12:01.454 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.454 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.454 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.454 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.454 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.454 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.454 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.454 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.454 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.454 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.454 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.454 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.454 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_pool 2024-03-24T12:12:01.454 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.455 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.455 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.455 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.455 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81884 2024-03-24T12:12:01.455 INFO:tasks.workunit.client.0.smithi006.stdout:test api_pool on pid 81884 2024-03-24T12:12:01.459 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_pool on pid 81884' 2024-03-24T12:12:01.459 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=81884 2024-03-24T12:12:01.459 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.460 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.460 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.460 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_snapshots 2024-03-24T12:12:01.460 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_snapshots' 2024-03-24T12:12:01.460 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-03-24T12:12:01.460 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.460 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.460 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.460 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.460 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.460 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.460 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.460 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.461 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.461 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.461 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.461 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.461 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.461 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/81807/exe 2024-03-24T12:12:01.461 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_snapshots 2024-03-24T12:12:01.461 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.461 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.461 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.461 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.461 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.461 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.462 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.462 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.462 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.462 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.462 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.462 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.462 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.462 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.462 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.462 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.462 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.462 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.462 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.462 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.463 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.463 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.463 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.463 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.463 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.463 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.463 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.463 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.463 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.463 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.463 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.463 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.463 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.463 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.464 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.464 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.464 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.464 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.464 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.464 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.464 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.464 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.464 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.464 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_snapshots 2024-03-24T12:12:01.464 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.464 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.464 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.464 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.464 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81901 2024-03-24T12:12:01.465 INFO:tasks.workunit.client.0.smithi006.stdout:test api_snapshots on pid 81901 2024-03-24T12:12:01.469 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_snapshots on pid 81901' 2024-03-24T12:12:01.469 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.469 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=81901 2024-03-24T12:12:01.469 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.469 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.469 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.469 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.469 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.469 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.469 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-03-24T12:12:01.469 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.470 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.470 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.470 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.470 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.470 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.470 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.470 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_lock_pp: /' 2024-03-24T12:12:01.470 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.470 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_snapshots_pp 2024-03-24T12:12:01.470 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.470 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.470 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_snapshots_pp' 2024-03-24T12:12:01.470 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.471 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.471 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.471 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.471 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.471 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-03-24T12:12:01.471 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.471 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.471 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.471 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.471 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.471 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.471 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.471 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.472 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.472 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.472 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.472 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.472 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.472 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.472 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.472 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.472 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.472 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.472 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.472 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.472 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.472 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.473 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.473 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.473 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.473 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.473 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.473 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.473 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.473 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.473 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.473 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.473 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.473 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.473 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.474 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.474 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.474 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.474 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.474 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.474 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.474 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.474 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.474 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.474 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.474 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.474 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.475 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.475 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.475 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.475 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.475 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.475 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2024-03-24T12:12:01.475 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.475 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.475 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.475 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.475 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.475 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.475 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.475 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.476 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.476 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.476 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.476 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.476 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.476 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_snapshots_pp 2024-03-24T12:12:01.476 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.476 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.476 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.476 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.477 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.477 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.477 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.477 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.477 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/81789/exe 2024-03-24T12:12:01.477 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.477 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.477 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.477 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.477 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.477 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.477 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.477 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.477 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.477 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.478 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.478 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.478 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.478 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.478 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.478 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.478 INFO:tasks.workunit.client.0.smithi006.stdout:test api_snapshots_pp on pid 81937 2024-03-24T12:12:01.482 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_snapshots_pp 2024-03-24T12:12:01.482 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81937 2024-03-24T12:12:01.482 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_snapshots_pp on pid 81937' 2024-03-24T12:12:01.482 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=81937 2024-03-24T12:12:01.482 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.482 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.482 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.482 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.483 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.483 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.483 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.483 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.483 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.483 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.483 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.483 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.483 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.483 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.483 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.483 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.483 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.484 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.484 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.484 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.484 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.484 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-03-24T12:12:01.484 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_lock: /' 2024-03-24T12:12:01.484 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.484 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.484 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.484 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.484 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.484 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.484 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_stat 2024-03-24T12:12:01.484 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_stat' 2024-03-24T12:12:01.484 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.485 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_lock.log 2024-03-24T12:12:01.485 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.485 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.485 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.485 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.485 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.485 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.485 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.485 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.485 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.485 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.485 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.485 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.486 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.486 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.486 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.486 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.486 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.486 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.486 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.486 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.486 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.486 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.486 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.486 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.486 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.486 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.487 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.487 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.487 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.487 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.487 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.487 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.487 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.487 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.487 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.487 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.487 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/81836/exe 2024-03-24T12:12:01.487 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.487 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.488 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.488 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.488 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.488 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.488 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.488 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.488 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.488 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.488 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.488 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.488 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.488 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.488 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.489 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.489 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.489 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.489 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.489 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.489 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.489 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.489 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.489 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.489 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.489 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.489 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.489 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.490 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.490 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.490 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.490 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.490 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.490 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.490 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.490 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.490 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.490 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.490 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2024-03-24T12:12:01.490 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.490 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.490 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.491 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.491 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.491 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.491 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.491 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.491 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.491 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.491 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.491 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.491 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.491 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.491 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.491 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.491 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.492 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.492 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.492 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.492 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.492 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.492 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.492 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.492 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.492 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.492 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.492 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.492 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.493 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.493 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_misc: /' 2024-03-24T12:12:01.493 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.493 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.493 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.493 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.493 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.493 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.493 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.493 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.493 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.493 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.493 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2024-03-24T12:12:01.494 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_misc.log 2024-03-24T12:12:01.494 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_stat 2024-03-24T12:12:01.494 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.494 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.494 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_stat 2024-03-24T12:12:01.494 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.494 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.494 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.494 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.494 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.494 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.494 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.494 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.494 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.494 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.495 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.495 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82009 2024-03-24T12:12:01.495 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_stat on pid 82009' 2024-03-24T12:12:01.495 INFO:tasks.workunit.client.0.smithi006.stdout:test api_stat on pid 82009 2024-03-24T12:12:01.499 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82009 2024-03-24T12:12:01.500 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.500 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.500 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.500 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.500 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.500 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.500 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.500 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.500 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.500 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_stat_pp 2024-03-24T12:12:01.500 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_stat_pp' 2024-03-24T12:12:01.500 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.500 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.501 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.501 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.501 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.501 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.501 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.501 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.501 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.501 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.501 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.501 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.501 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.501 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.501 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.501 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.502 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.502 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.502 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.502 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.502 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_stat_pp 2024-03-24T12:12:01.502 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-03-24T12:12:01.502 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.502 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.502 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.502 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.502 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.502 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.502 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.503 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.503 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.503 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.503 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.503 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.503 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.503 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.503 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.503 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.503 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.503 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.503 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.503 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.504 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.504 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.504 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.504 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.504 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.504 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.504 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.504 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.504 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.504 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.504 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.504 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.504 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.504 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.504 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.505 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.505 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.505 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.505 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.505 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.505 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.505 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.505 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.505 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.505 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.505 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.505 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.505 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.506 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.506 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.506 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.506 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.506 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.506 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.506 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.506 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.506 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.506 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.506 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.506 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.506 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.507 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.507 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.507 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.507 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.507 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.507 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.507 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.507 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.507 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.507 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.507 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.507 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.507 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.507 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.508 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.508 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.508 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.508 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.508 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.508 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.508 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.508 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.508 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.508 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.508 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.508 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.508 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.509 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.509 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.509 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.509 INFO:tasks.workunit.client.0.smithi006.stdout:test api_stat_pp on pid 82031 2024-03-24T12:12:01.512 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.512 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.512 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.512 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.512 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.512 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.512 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.512 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.512 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.512 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.512 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.512 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.513 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.513 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.513 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.513 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_stat_pp 2024-03-24T12:12:01.513 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.513 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.513 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.513 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.513 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.513 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.513 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82031 2024-03-24T12:12:01.513 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_stat_pp on pid 82031' 2024-03-24T12:12:01.513 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82031 2024-03-24T12:12:01.514 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.514 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.514 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.514 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.514 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.514 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.514 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.514 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_watch_notify 2024-03-24T12:12:01.514 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.514 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.514 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.514 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.514 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.515 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_watch_notify' 2024-03-24T12:12:01.515 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.515 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.515 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.515 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.515 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.515 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.515 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.515 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.515 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.515 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.515 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.515 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.516 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/81884/exe 2024-03-24T12:12:01.516 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.516 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.516 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.516 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.516 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.516 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.516 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.516 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.516 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.516 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.516 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.516 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_watch_notify 2024-03-24T12:12:01.517 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.517 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.517 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.517 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.517 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.517 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.517 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.517 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.517 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.517 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.517 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.517 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.517 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.517 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.517 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.518 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.518 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.518 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.518 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.518 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.518 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.518 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.518 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.518 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.518 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.518 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.518 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.518 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.518 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.518 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.519 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.519 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.519 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.519 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.519 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.519 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.519 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.519 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.519 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.519 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.519 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.519 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.519 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.519 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.519 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.520 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.520 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.520 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.520 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.520 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.520 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.520 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.520 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.520 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.520 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.520 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.520 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.520 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.520 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.520 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.521 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.521 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.521 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.521 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.521 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.521 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/81861/exe 2024-03-24T12:12:01.521 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.521 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.521 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.521 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.521 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.521 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.521 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.521 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.521 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.522 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.522 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.522 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.522 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.522 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.522 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.522 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.522 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.522 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.522 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.522 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.522 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.522 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.522 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.522 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.523 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.523 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.523 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.523 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_pool: /' 2024-03-24T12:12:01.523 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.523 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/81873/exe 2024-03-24T12:12:01.523 INFO:tasks.workunit.client.0.smithi006.stdout:test api_watch_notify on pid 82054 2024-03-24T12:12:01.528 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.528 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.528 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_watch_notify 2024-03-24T12:12:01.528 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.528 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.528 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.528 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.528 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.529 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82054 2024-03-24T12:12:01.529 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_watch_notify on pid 82054' 2024-03-24T12:12:01.529 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82054 2024-03-24T12:12:01.529 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.529 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.529 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.529 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.529 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.529 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.529 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.529 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_watch_notify_pp 2024-03-24T12:12:01.529 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_watch_notify_pp' 2024-03-24T12:12:01.530 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_misc_pp: /' 2024-03-24T12:12:01.530 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.530 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.530 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.530 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.530 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.530 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.530 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.530 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-03-24T12:12:01.530 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.530 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.530 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.530 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-03-24T12:12:01.530 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.531 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.531 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.531 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.531 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.531 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/81901/exe 2024-03-24T12:12:01.531 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-03-24T12:12:01.531 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.531 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.531 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_pool.log 2024-03-24T12:12:01.531 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.531 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.531 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.531 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.531 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.532 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.532 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.532 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.532 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.532 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.532 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2024-03-24T12:12:01.532 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.532 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.532 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.532 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.532 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.532 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.532 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.532 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.533 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.533 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.533 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.533 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_tier_pp: /' 2024-03-24T12:12:01.533 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.533 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.533 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.533 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.533 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.533 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.533 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.534 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.534 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.534 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.534 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.534 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.534 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.534 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.534 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.534 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.534 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.534 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.534 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.535 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.535 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.535 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.535 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.535 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.535 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_watch_notify_pp 2024-03-24T12:12:01.535 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2024-03-24T12:12:01.535 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_snapshots: /' 2024-03-24T12:12:01.535 INFO:tasks.workunit.client.0.smithi006.stdout:test api_watch_notify_pp on pid 82080 2024-03-24T12:12:01.538 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_watch_notify_pp 2024-03-24T12:12:01.538 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82080 2024-03-24T12:12:01.538 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_watch_notify_pp on pid 82080' 2024-03-24T12:12:01.539 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82080 2024-03-24T12:12:01.539 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.539 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.539 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2024-03-24T12:12:01.539 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.539 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.539 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2024-03-24T12:12:01.539 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.539 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.539 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.539 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.539 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.539 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.539 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.540 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.540 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.540 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.540 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.540 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.540 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.540 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.540 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.540 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.540 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.540 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.541 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.541 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.541 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.541 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.541 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.541 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.541 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.541 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.541 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.541 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.541 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.541 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.541 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.542 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.542 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.542 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.542 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.542 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.542 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.542 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.542 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.542 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.542 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.542 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.542 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.542 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.542 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.542 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.543 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.543 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.543 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.543 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.543 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.543 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.543 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.543 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.543 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.543 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.543 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.543 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.543 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.543 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.543 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.544 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.544 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.544 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.544 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-03-24T12:12:01.544 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-03-24T12:12:01.544 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_cmd 2024-03-24T12:12:01.544 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_cmd' 2024-03-24T12:12:01.544 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_cmd 2024-03-24T12:12:01.544 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.544 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.544 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.544 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.544 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.544 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.545 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.545 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.545 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-03-24T12:12:01.545 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.545 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.545 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.545 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.545 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.545 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.545 INFO:tasks.workunit.client.0.smithi006.stdout:test api_cmd on pid 82120 2024-03-24T12:12:01.553 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.554 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.554 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.554 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.554 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.554 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.554 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.554 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.554 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.554 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.554 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.554 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.554 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_cmd 2024-03-24T12:12:01.554 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.555 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.555 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.555 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.555 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.555 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82120 2024-03-24T12:12:01.555 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.555 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_cmd on pid 82120' 2024-03-24T12:12:01.555 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.555 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82120 2024-03-24T12:12:01.555 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.555 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.555 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.555 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.556 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.556 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.556 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.556 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.556 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.556 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.556 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.556 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.556 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_cmd_pp 2024-03-24T12:12:01.556 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.556 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.556 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.556 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.557 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.557 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_cmd_pp' 2024-03-24T12:12:01.557 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.557 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-03-24T12:12:01.557 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.557 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.557 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.557 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.557 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.557 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.557 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.557 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.557 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/81937/exe 2024-03-24T12:12:01.557 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.558 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.558 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.558 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.558 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.558 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.558 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.558 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.558 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.558 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.558 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.558 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.558 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.558 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.558 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.559 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.559 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.559 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.559 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.559 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.559 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.559 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.559 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.559 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.559 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.559 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.559 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.559 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.559 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.559 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.560 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.560 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.560 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.560 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.560 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.560 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.560 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.560 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.560 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.560 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.560 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.560 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.560 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.560 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.560 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.561 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.561 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.561 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.561 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.561 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.561 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.561 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.561 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.561 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.561 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.561 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.561 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.561 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.561 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.562 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.562 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.562 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.562 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.562 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.562 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.562 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.562 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.562 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.562 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.562 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.562 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.562 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.562 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.562 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.563 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.563 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82009/exe 2024-03-24T12:12:01.563 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.563 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.563 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.563 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.563 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.563 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.563 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.563 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.563 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.563 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.563 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.563 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.563 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.564 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.564 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.564 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.564 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.564 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.564 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.564 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.564 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.564 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.564 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.564 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.564 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.564 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.564 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.564 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.565 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.565 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.565 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.565 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.565 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.565 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.565 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.565 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.565 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.565 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.565 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.565 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.565 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.565 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.565 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.566 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2024-03-24T12:12:01.566 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.566 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.566 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.566 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.566 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.566 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.566 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.566 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.566 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.566 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.566 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.566 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.566 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.566 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.567 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.567 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.567 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.567 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.567 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-03-24T12:12:01.567 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.567 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.567 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.567 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.567 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.567 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.567 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.567 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.567 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.567 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.567 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.568 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.568 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.568 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.568 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.568 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.568 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.568 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.568 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.568 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.568 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-03-24T12:12:01.568 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_stat: /' 2024-03-24T12:12:01.568 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.568 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_stat.log 2024-03-24T12:12:01.568 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.568 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.569 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.569 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.569 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.569 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.569 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.569 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.569 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_cmd_pp 2024-03-24T12:12:01.569 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82031/exe 2024-03-24T12:12:01.569 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.569 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.569 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.569 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.569 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.569 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.569 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.570 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.570 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.570 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.570 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.570 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.570 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.570 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.570 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.570 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.570 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.570 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.570 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.570 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.570 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2024-03-24T12:12:01.571 INFO:tasks.workunit.client.0.smithi006.stdout:test api_cmd_pp on pid 82163 2024-03-24T12:12:01.576 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.577 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.577 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.577 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_cmd_pp 2024-03-24T12:12:01.577 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.577 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.577 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.577 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.577 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.577 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.577 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.577 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.577 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.577 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82163 2024-03-24T12:12:01.577 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_cmd_pp on pid 82163' 2024-03-24T12:12:01.578 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82163 2024-03-24T12:12:01.578 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.578 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.578 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.578 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.578 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.578 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.578 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.578 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2024-03-24T12:12:01.578 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.578 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.578 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.578 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.578 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.579 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.579 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.579 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_stat_pp: /' 2024-03-24T12:12:01.579 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.579 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.579 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.579 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.579 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.579 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.579 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.579 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.579 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.579 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.579 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.579 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.580 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.580 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.580 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.580 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.580 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.580 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.580 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.580 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.580 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.580 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82054/exe 2024-03-24T12:12:01.580 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.580 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.580 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.580 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.580 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.581 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.581 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.581 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.581 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.581 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.581 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.581 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.581 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.581 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.581 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.581 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.581 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.581 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.581 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.581 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.582 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.582 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.582 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.582 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.582 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.582 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.582 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.582 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.582 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.582 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.582 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.582 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_watch_notify: /' 2024-03-24T12:12:01.582 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_service 2024-03-24T12:12:01.582 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_service' 2024-03-24T12:12:01.582 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-03-24T12:12:01.583 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2024-03-24T12:12:01.583 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-03-24T12:12:01.583 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-03-24T12:12:01.583 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.583 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.583 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.583 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.583 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.583 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.583 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.583 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.583 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.583 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.583 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.583 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.584 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.584 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.584 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.584 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.584 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.584 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.584 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.584 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.584 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.584 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.584 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.584 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.584 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.584 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.584 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.585 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.585 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.585 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.585 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.585 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.585 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.585 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.585 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.585 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.585 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.585 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.585 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.585 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.585 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.585 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.586 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.586 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.586 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.586 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.586 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.586 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.586 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.586 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.586 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.586 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.586 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.586 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.586 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.586 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.586 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.587 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.587 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.587 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.587 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.587 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.587 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.587 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.587 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.587 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.587 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.587 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.587 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_service 2024-03-24T12:12:01.587 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.587 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.587 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.588 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.588 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.588 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.588 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.588 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.588 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.588 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.588 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.588 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.588 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.588 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.588 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.588 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.588 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.588 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.589 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.589 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.589 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.589 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.589 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.589 INFO:tasks.workunit.client.0.smithi006.stdout:test api_service on pid 82233 2024-03-24T12:12:01.592 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_service 2024-03-24T12:12:01.592 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82233 2024-03-24T12:12:01.593 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_service on pid 82233' 2024-03-24T12:12:01.593 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82233 2024-03-24T12:12:01.593 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.593 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.593 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.593 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.593 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.593 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.593 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.593 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.593 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.593 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.593 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.593 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.594 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.594 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.594 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.594 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.594 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-03-24T12:12:01.594 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.594 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.594 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_service_pp 2024-03-24T12:12:01.594 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_service_pp' 2024-03-24T12:12:01.594 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.594 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.594 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.594 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.594 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.595 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.595 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.595 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.595 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.595 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.595 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.595 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.595 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.595 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.595 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.595 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.595 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.595 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.595 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.595 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.596 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.596 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.596 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.596 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.596 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.596 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.596 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.596 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.596 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.596 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.596 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.596 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.596 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.596 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.597 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.597 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.597 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.597 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.597 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.597 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.597 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.597 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.597 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.597 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.597 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.597 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.597 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.597 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.597 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.598 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.598 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.598 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.598 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.598 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.598 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.598 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.598 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.598 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.598 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.598 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.598 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.601 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.602 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.602 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.602 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.602 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.602 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.602 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.602 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.603 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.603 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.603 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.603 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82080/exe 2024-03-24T12:12:01.603 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.603 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.603 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.603 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.603 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.604 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.604 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.604 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.604 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.604 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.604 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.604 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_service_pp 2024-03-24T12:12:01.604 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2024-03-24T12:12:01.604 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.604 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82120/exe 2024-03-24T12:12:01.605 INFO:tasks.workunit.client.0.smithi006.stdout:test api_service_pp on pid 82284 2024-03-24T12:12:01.608 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.608 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.608 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.608 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.608 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.608 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.608 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.608 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.609 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.609 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.609 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.609 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.609 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.609 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.609 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.609 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.609 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.609 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.609 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_service_pp 2024-03-24T12:12:01.609 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.609 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.609 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82284 2024-03-24T12:12:01.609 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_service_pp on pid 82284' 2024-03-24T12:12:01.610 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82284 2024-03-24T12:12:01.610 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.610 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.610 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.610 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.610 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.610 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.610 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.610 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.610 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.610 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.610 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.610 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.610 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.611 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.611 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.611 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-03-24T12:12:01.611 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_cmd: /' 2024-03-24T12:12:01.611 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.611 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.611 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.611 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.611 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.611 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.611 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.611 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.611 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.611 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.612 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.612 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.612 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.612 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.612 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.612 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.612 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_c_write_operations 2024-03-24T12:12:01.612 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.613 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.613 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.613 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.613 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_c_write_operations' 2024-03-24T12:12:01.613 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.613 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.613 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2024-03-24T12:12:01.613 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.613 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.613 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_c_write_operations 2024-03-24T12:12:01.613 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.613 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.613 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.613 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.613 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.614 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.614 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.614 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.614 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.614 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-03-24T12:12:01.614 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.614 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.614 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.614 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.614 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.614 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-03-24T12:12:01.614 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.614 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.614 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.615 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_c_write_operations 2024-03-24T12:12:01.615 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.615 INFO:tasks.workunit.client.0.smithi006.stdout:test api_c_write_operations on pid 82319 2024-03-24T12:12:01.618 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82319 2024-03-24T12:12:01.619 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_c_write_operations on pid 82319' 2024-03-24T12:12:01.619 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82319 2024-03-24T12:12:01.619 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.619 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.619 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_c_read_operations 2024-03-24T12:12:01.619 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-03-24T12:12:01.619 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_c_read_operations' 2024-03-24T12:12:01.619 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.619 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.619 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.619 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.619 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.619 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.619 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.620 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.620 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.620 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_cmd.log 2024-03-24T12:12:01.620 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.620 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.620 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.620 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.620 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.620 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.620 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.620 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.620 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.620 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.620 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.621 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.621 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.621 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.621 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.621 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.621 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.621 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.621 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.621 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.621 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.621 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.621 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.621 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.622 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.622 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.622 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.622 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.622 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.622 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.622 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.622 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.622 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_c_read_operations 2024-03-24T12:12:01.622 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.623 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.623 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.623 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.623 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.623 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.623 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.623 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.623 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.623 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.623 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.623 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.623 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.624 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.624 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.624 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.624 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.624 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.624 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.624 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.624 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.624 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.624 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.624 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.624 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.624 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.624 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.625 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.625 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.625 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.625 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.625 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.625 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.625 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.625 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.625 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.625 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.625 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.625 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.625 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.625 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.626 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.626 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.626 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.626 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.626 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.626 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.626 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.626 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.626 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.626 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.626 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.627 INFO:tasks.workunit.client.0.smithi006.stdout:test api_c_read_operations on pid 82379 2024-03-24T12:12:01.629 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_c_read_operations 2024-03-24T12:12:01.629 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82379 2024-03-24T12:12:01.629 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_c_read_operations on pid 82379' 2024-03-24T12:12:01.629 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82379 2024-03-24T12:12:01.629 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.629 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.629 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s api_cls_remote_reads 2024-03-24T12:12:01.629 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.629 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.629 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.629 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.629 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.630 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' api_cls_remote_reads' 2024-03-24T12:12:01.630 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.630 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.630 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.630 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.630 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.630 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.630 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-03-24T12:12:01.630 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.630 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.630 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.630 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.630 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo api_cls_remote_reads 2024-03-24T12:12:01.630 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.631 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.631 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.631 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.631 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.631 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.631 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.631 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.631 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.631 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.631 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.631 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.631 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.631 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.631 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.632 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.632 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.632 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.632 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.632 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.632 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.632 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.632 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.632 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.632 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.633 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.633 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.633 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.633 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.633 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.634 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.634 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.634 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.634 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.634 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.634 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.634 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.634 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.634 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.634 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.634 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.634 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.634 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.634 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.634 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.635 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.635 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.635 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.635 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.635 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.635 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.635 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.635 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.635 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.635 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.635 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.635 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.635 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.635 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.635 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.635 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.636 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.636 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.636 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.636 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.636 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.636 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.636 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.636 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.636 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.636 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.636 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=api_cls_remote_reads 2024-03-24T12:12:01.636 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.636 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.637 INFO:tasks.workunit.client.0.smithi006.stdout:test api_cls_remote_reads on pid 82403 2024-03-24T12:12:01.639 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.639 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.639 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.639 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.639 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82403 2024-03-24T12:12:01.639 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test api_cls_remote_reads on pid 82403' 2024-03-24T12:12:01.639 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82403 2024-03-24T12:12:01.639 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.639 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.640 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.640 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.640 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.640 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.640 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.640 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82233/exe 2024-03-24T12:12:01.640 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.640 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cls_remote_reads.xml 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2024-03-24T12:12:01.640 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.640 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.640 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.640 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.640 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.641 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.641 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.641 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.641 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.641 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.641 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.641 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.641 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.641 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.641 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.641 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.641 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.641 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82163/exe 2024-03-24T12:12:01.641 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.641 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.642 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.642 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s list_parallel 2024-03-24T12:12:01.642 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' list_parallel' 2024-03-24T12:12:01.642 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.642 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.642 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.642 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.642 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.642 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.642 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.642 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.642 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.642 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.642 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.642 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.643 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.643 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.643 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.643 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.643 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.643 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.643 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.643 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.643 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.643 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.643 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.643 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.643 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.643 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.644 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.644 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.644 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.644 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.644 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo list_parallel 2024-03-24T12:12:01.644 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.644 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.644 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.644 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.644 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.644 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.644 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.644 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.644 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.644 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.645 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.645 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.645 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.645 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.645 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.645 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.645 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.645 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.645 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.645 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.645 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.645 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.645 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.645 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.645 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2024-03-24T12:12:01.645 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.646 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.646 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.646 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.646 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.646 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.646 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.646 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.646 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.646 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.646 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.646 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.646 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.646 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.646 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.646 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.647 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.647 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.647 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.647 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.647 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.647 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.647 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.647 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.647 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.647 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_service: /' 2024-03-24T12:12:01.647 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.647 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.647 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.647 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.647 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.647 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.648 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.648 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.648 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.648 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-03-24T12:12:01.648 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2024-03-24T12:12:01.648 INFO:tasks.workunit.client.0.smithi006.stdout:test list_parallel on pid 82425 2024-03-24T12:12:01.650 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=list_parallel 2024-03-24T12:12:01.650 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82425 2024-03-24T12:12:01.650 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test list_parallel on pid 82425' 2024-03-24T12:12:01.650 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82425 2024-03-24T12:12:01.651 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.651 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.651 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s open_pools_parallel 2024-03-24T12:12:01.651 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.651 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' open_pools_parallel' 2024-03-24T12:12:01.651 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-03-24T12:12:01.651 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-03-24T12:12:01.651 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.651 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.651 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.651 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.651 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.651 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.651 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.651 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.651 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.652 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.652 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.652 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.652 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.652 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.652 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.652 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.652 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.652 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.652 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.652 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.652 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.652 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.652 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.653 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.653 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.653 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.653 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.653 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.653 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.653 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.653 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.653 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.653 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.653 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.653 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.653 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.653 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.653 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.653 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.654 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.654 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.654 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.654 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.654 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.654 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.654 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.654 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.654 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.654 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.654 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.654 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.654 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.654 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.654 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.655 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.655 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.655 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.655 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.655 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.655 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.655 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.655 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.655 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.655 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.655 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.655 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.655 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo open_pools_parallel 2024-03-24T12:12:01.655 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.655 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.656 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.656 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.656 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.656 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.656 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.656 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.656 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.656 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.656 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82319/exe 2024-03-24T12:12:01.656 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.656 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.656 INFO:tasks.workunit.client.0.smithi006.stdout:test open_pools_parallel on pid 82443 2024-03-24T12:12:01.661 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.661 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.661 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.661 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.661 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.661 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.661 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.661 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.661 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.661 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.661 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=open_pools_parallel 2024-03-24T12:12:01.662 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82443 2024-03-24T12:12:01.662 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test open_pools_parallel on pid 82443' 2024-03-24T12:12:01.662 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82443 2024-03-24T12:12:01.662 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-24T12:12:01.662 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.662 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s delete_pools_parallel 2024-03-24T12:12:01.662 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' delete_pools_parallel' 2024-03-24T12:12:01.662 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.662 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.662 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-03-24T12:12:01.662 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_service.log 2024-03-24T12:12:01.662 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.662 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.662 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.663 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-03-24T12:12:01.663 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-03-24T12:12:01.663 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.663 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.663 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.663 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.663 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.663 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.663 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.663 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.663 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.663 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2024-03-24T12:12:01.663 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.663 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.664 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.664 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.664 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.664 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.664 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.664 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.664 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.664 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.664 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.664 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.664 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.664 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.664 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.665 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.665 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.665 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo delete_pools_parallel 2024-03-24T12:12:01.665 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.665 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.665 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.665 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.665 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.666 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.666 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.666 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.666 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.666 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.666 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.666 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.666 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.666 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.666 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.666 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.666 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.666 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.666 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.667 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.667 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.667 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.667 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.667 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.667 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.667 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.667 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.667 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.667 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.667 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.667 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.667 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.667 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.667 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.668 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.668 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.668 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.668 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.668 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.668 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.668 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.668 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.668 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.668 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.668 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.668 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.668 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.668 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.668 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.669 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.669 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.669 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.669 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.669 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.669 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.669 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.669 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.669 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.669 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.669 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.670 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.670 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.670 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.670 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.670 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.670 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.670 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.670 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.670 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82284/exe 2024-03-24T12:12:01.670 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.670 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.670 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.670 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.670 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.671 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.671 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.671 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.671 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.671 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.671 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.671 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.671 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.671 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.671 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.671 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.671 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.671 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.671 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.671 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.672 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.672 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=delete_pools_parallel 2024-03-24T12:12:01.672 INFO:tasks.workunit.client.0.smithi006.stdout:test delete_pools_parallel on pid 82480 2024-03-24T12:12:01.675 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82480 2024-03-24T12:12:01.675 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.675 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test delete_pools_parallel on pid 82480' 2024-03-24T12:12:01.676 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82480 2024-03-24T12:12:01.676 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.676 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-24T12:12:01.676 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.676 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.676 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.676 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.676 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.676 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.677 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.677 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.677 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.677 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.677 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.677 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.677 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.677 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.677 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.677 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.677 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.677 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.677 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.677 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.678 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.678 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.678 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.678 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.678 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-03-24T12:12:01.678 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.678 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.678 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.678 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.678 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.679 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.679 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.679 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.679 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.679 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.679 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.679 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.679 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.679 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.679 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.679 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.679 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.680 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-03-24T12:12:01.680 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_service_pp: /' 2024-03-24T12:12:01.680 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.680 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.680 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.680 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.680 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.680 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.680 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.680 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.680 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.680 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.680 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.680 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.680 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.681 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.681 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.681 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.681 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.681 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.681 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.681 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.681 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.681 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.681 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.681 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.681 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.681 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.681 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s cls 2024-03-24T12:12:01.681 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' cls' 2024-03-24T12:12:01.682 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2024-03-24T12:12:01.682 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.682 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.682 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.682 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.682 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.682 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.682 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.682 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.682 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.682 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.682 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.682 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.682 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.682 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.683 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.683 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.683 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.683 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.683 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.683 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.683 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.683 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.683 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.683 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.683 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.683 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.683 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.683 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.683 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.684 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.684 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.684 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.684 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.684 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.684 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.684 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.684 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.684 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.684 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.684 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.684 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo cls 2024-03-24T12:12:01.684 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82403/exe 2024-03-24T12:12:01.684 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.684 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.685 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.685 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.685 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.685 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.685 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.685 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.685 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.685 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.685 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.685 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.685 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.685 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.685 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.685 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.686 INFO:tasks.workunit.client.0.smithi006.stdout:test cls on pid 82514 2024-03-24T12:12:01.688 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.688 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.688 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.688 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.688 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.688 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.688 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=cls 2024-03-24T12:12:01.688 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.688 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82514 2024-03-24T12:12:01.688 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.688 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test cls on pid 82514' 2024-03-24T12:12:01.688 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82514 2024-03-24T12:12:01.688 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-24T12:12:01.689 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.689 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.689 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.689 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2024-03-24T12:12:01.689 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2024-03-24T12:12:01.689 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.689 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.689 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.689 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.689 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.689 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s cmd 2024-03-24T12:12:01.689 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.689 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' cmd' 2024-03-24T12:12:01.690 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.690 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.690 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.690 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.690 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.690 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.690 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.690 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.690 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.690 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.690 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.690 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.690 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.690 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.691 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.691 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.691 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.691 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.691 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.691 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.691 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.691 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.691 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.691 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.691 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.691 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.691 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.691 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.691 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.692 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.692 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.692 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.692 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.692 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.692 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.692 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.692 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.692 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.692 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.692 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.692 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo cmd 2024-03-24T12:12:01.692 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.692 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cls 2>&1 | tee ceph_test_neorados_cls.log | sed "s/^/ cls: /"' 2024-03-24T12:12:01.693 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.693 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.693 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.693 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.693 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.693 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.693 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.693 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.693 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.693 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.693 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.693 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.693 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.693 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.693 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.694 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.694 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.694 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.694 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.694 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.694 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.694 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.694 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.694 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.694 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.694 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.694 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82425/exe 2024-03-24T12:12:01.694 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82379/exe 2024-03-24T12:12:01.695 INFO:tasks.workunit.client.0.smithi006.stdout:test cmd on pid 82535 2024-03-24T12:12:01.699 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=cmd 2024-03-24T12:12:01.699 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82535 2024-03-24T12:12:01.699 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test cmd on pid 82535' 2024-03-24T12:12:01.699 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82535 2024-03-24T12:12:01.699 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-24T12:12:01.699 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.699 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.699 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.701 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s handler_error 2024-03-24T12:12:01.701 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.701 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.701 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.701 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' handler_error' 2024-03-24T12:12:01.701 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cmd 2>&1 | tee ceph_test_neorados_cmd.log | sed "s/^/ cmd: /"' 2024-03-24T12:12:01.701 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cls_remote_reads.xml 2024-03-24T12:12:01.701 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ list_parallel: /' 2024-03-24T12:12:01.701 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.701 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.701 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.701 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_list_parallel.log 2024-03-24T12:12:01.701 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.701 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.701 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.702 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.702 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.702 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.702 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.702 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.702 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.702 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.702 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.702 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.702 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.702 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.702 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.702 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.702 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.702 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.703 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.703 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.703 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.703 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.703 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.703 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.703 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.703 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.703 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.703 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.703 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.703 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.703 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.703 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.703 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.704 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.704 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.704 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2024-03-24T12:12:01.704 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-03-24T12:12:01.704 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.704 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.704 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.704 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.704 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.704 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.704 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.704 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.704 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.704 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.704 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.705 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.705 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2024-03-24T12:12:01.705 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.706 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.706 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.706 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.706 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.706 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.706 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo handler_error 2024-03-24T12:12:01.706 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.706 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82443/exe 2024-03-24T12:12:01.707 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.707 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.707 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.708 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.708 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.708 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.708 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.708 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.708 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.708 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.708 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-03-24T12:12:01.708 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.708 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.708 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.708 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.708 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.708 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.708 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.709 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.709 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.709 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=handler_error 2024-03-24T12:12:01.709 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.709 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.710 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82559 2024-03-24T12:12:01.710 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test handler_error on pid 82559' 2024-03-24T12:12:01.710 INFO:tasks.workunit.client.0.smithi006.stdout:test handler_error on pid 82559 2024-03-24T12:12:01.713 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82559 2024-03-24T12:12:01.714 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.714 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-24T12:12:01.714 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.714 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.714 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.714 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.714 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.714 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.714 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.714 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.714 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.714 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.714 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.714 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.715 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.715 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.715 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.715 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.715 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.715 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.715 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.715 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.715 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.715 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.715 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.715 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.715 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.715 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.716 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.716 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.716 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.716 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_handler_error 2>&1 | tee ceph_test_neorados_handler_error.log | sed "s/^/ handler_error: /"' 2024-03-24T12:12:01.716 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.716 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.716 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-03-24T12:12:01.716 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.716 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.716 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.716 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.716 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.716 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.716 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.716 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.717 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.717 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.717 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.717 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.717 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.717 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.717 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.717 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.717 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.717 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.717 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.717 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.717 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.717 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.717 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.718 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-03-24T12:12:01.718 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.718 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.718 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.718 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.718 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.718 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.718 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.718 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.718 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.718 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.718 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.718 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.718 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.719 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.719 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.719 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.719 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.719 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.719 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.719 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.719 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s io 2024-03-24T12:12:01.719 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2024-03-24T12:12:01.719 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' io' 2024-03-24T12:12:01.719 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.719 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.719 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.719 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.719 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.720 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.720 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.720 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.720 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.720 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.720 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.720 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.720 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.720 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.720 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.720 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.720 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.720 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.720 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.720 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.721 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo io 2024-03-24T12:12:01.721 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.721 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.721 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.721 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.721 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.721 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.721 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.721 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.721 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.721 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.721 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.721 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.721 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.721 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.722 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.722 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.722 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.722 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.722 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.722 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.722 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.722 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.722 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.722 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.722 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.722 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.722 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.722 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.722 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.723 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.723 INFO:tasks.workunit.client.0.smithi006.stdout:test io on pid 82587 2024-03-24T12:12:01.724 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=io 2024-03-24T12:12:01.724 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82587 2024-03-24T12:12:01.724 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test io on pid 82587' 2024-03-24T12:12:01.724 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82587 2024-03-24T12:12:01.725 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-24T12:12:01.725 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.725 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_io 2>&1 | tee ceph_test_neorados_io.log | sed "s/^/ io: /"' 2024-03-24T12:12:01.725 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.725 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.725 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.725 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.725 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.725 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.725 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.725 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.725 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.725 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.725 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.725 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.726 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.726 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.726 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.726 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.726 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.726 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s ec_io 2024-03-24T12:12:01.726 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' ec_io' 2024-03-24T12:12:01.726 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.726 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.726 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo ec_io 2024-03-24T12:12:01.726 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.726 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.726 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.726 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.726 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.727 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.727 INFO:tasks.workunit.client.0.smithi006.stdout:test ec_io on pid 82614 2024-03-24T12:12:01.732 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=ec_io 2024-03-24T12:12:01.732 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82614 2024-03-24T12:12:01.732 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test ec_io on pid 82614' 2024-03-24T12:12:01.732 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82614 2024-03-24T12:12:01.732 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-24T12:12:01.732 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.732 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.732 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_io 2>&1 | tee ceph_test_neorados_ec_io.log | sed "s/^/ ec_io: /"' 2024-03-24T12:12:01.732 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.733 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.733 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.733 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.733 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.733 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.733 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.733 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.733 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.733 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.733 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.733 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.733 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.733 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.733 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.734 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.734 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.734 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.734 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.734 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.734 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.734 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.734 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.734 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.734 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.734 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.734 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.734 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.734 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.735 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.735 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.735 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.735 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.735 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s list 2024-03-24T12:12:01.735 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' list' 2024-03-24T12:12:01.735 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.735 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.735 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.735 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.735 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.735 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.735 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.735 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.736 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.736 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.736 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.736 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.736 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.736 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.736 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.736 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.736 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.736 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.736 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.736 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.736 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.736 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.736 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.737 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.737 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.737 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.737 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.737 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.737 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.737 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.737 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.737 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.737 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.737 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.737 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.737 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.737 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.737 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.738 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.738 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.738 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.738 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.738 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.738 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.738 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.738 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.738 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.738 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.738 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.738 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.738 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.738 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.739 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.739 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.739 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.739 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.739 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.739 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.739 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.739 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.739 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.739 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.739 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.739 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.739 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.739 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.740 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.740 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.740 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.740 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.740 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.740 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.740 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.740 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.740 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.740 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.740 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.740 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.740 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.740 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.740 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.741 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.741 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.741 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.741 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.741 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.741 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.741 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.741 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.741 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.741 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.741 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.741 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.741 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.741 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.741 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.742 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.742 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.742 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.742 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.742 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.742 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.742 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.742 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.742 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.742 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.742 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.742 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.742 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.743 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.743 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.743 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.743 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.743 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.743 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.743 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.743 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.743 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.743 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.743 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.743 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.743 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.743 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.743 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.744 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.744 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.744 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.744 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.744 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.744 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.744 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.744 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.744 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.744 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.744 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.744 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.744 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.744 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.744 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.745 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.745 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.745 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.745 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.745 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.745 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.745 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82480/exe 2024-03-24T12:12:01.745 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.745 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.745 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.745 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.745 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.745 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.745 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.745 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.746 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.746 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.746 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.746 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.746 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.746 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.746 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.746 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.746 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.746 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.746 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.746 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.746 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.746 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.746 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.747 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.747 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.747 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.747 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.747 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.747 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82514/exe 2024-03-24T12:12:01.747 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.747 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo list 2024-03-24T12:12:01.747 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.747 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.747 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.747 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.747 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.747 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.747 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.748 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.748 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.748 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.748 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.748 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.748 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2024-03-24T12:12:01.748 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.748 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.748 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.748 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.748 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.748 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.748 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.748 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.748 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.748 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.749 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.749 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.749 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.749 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.749 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.749 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.749 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.749 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.749 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.749 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.749 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.749 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.749 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.749 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.749 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.750 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.750 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.750 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.750 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.750 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.750 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.750 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=list 2024-03-24T12:12:01.750 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82685 2024-03-24T12:12:01.750 INFO:tasks.workunit.client.0.smithi006.stdout:test list on pid 82685 2024-03-24T12:12:01.756 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test list on pid 82685' 2024-03-24T12:12:01.756 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82685 2024-03-24T12:12:01.756 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-24T12:12:01.756 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.756 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-03-24T12:12:01.757 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.757 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.757 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.757 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.757 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.757 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.757 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.757 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.757 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.757 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-03-24T12:12:01.757 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_list 2>&1 | tee ceph_test_neorados_list.log | sed "s/^/ list: /"' 2024-03-24T12:12:01.757 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s ec_list 2024-03-24T12:12:01.757 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' ec_list' 2024-03-24T12:12:01.757 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_neorados_cls 2024-03-24T12:12:01.757 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_neorados_cls.log 2024-03-24T12:12:01.758 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.758 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.758 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.758 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.758 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.758 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.758 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.758 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.758 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.758 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.758 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.758 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.758 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.758 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.759 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.759 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.759 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.759 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.759 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.759 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.759 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.759 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.759 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.759 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ cls: /' 2024-03-24T12:12:01.759 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.759 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.759 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo ec_list 2024-03-24T12:12:01.759 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.759 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.760 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.760 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.760 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.760 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.760 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.760 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.760 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.760 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.760 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.760 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.760 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.760 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.760 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.760 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.760 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.760 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.761 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.761 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.761 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.761 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.761 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.761 INFO:tasks.workunit.client.0.smithi006.stdout:test ec_list on pid 82707 2024-03-24T12:12:01.767 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=ec_list 2024-03-24T12:12:01.767 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82707 2024-03-24T12:12:01.767 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test ec_list on pid 82707' 2024-03-24T12:12:01.767 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82707 2024-03-24T12:12:01.767 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-24T12:12:01.767 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.768 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s misc 2024-03-24T12:12:01.768 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.768 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.768 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' misc' 2024-03-24T12:12:01.768 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.768 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.768 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.768 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_list 2>&1 | tee ceph_test_neorados_ec_list.log | sed "s/^/ ec_list: /"' 2024-03-24T12:12:01.768 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.768 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.768 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.768 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.768 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.768 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.768 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.769 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.769 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.769 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.769 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.769 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.769 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.769 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.769 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.769 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.769 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.769 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.769 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.769 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.769 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.769 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.770 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.770 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.770 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.770 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.770 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.770 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.770 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.770 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.770 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.770 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.770 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.770 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.770 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.771 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.771 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.771 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.771 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.771 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.771 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.771 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.771 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.771 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.771 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.771 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.771 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.771 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.771 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.771 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.772 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.772 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.772 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.772 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.772 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.772 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.772 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.772 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.772 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.772 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.772 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.772 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.772 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.772 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.772 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.773 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.773 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.773 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.773 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.773 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.773 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.773 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.773 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.773 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.773 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.773 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.773 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.773 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.773 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.773 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.774 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.774 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.774 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.774 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.774 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.774 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.774 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.774 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.774 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.774 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.774 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.774 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.774 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82587/exe 2024-03-24T12:12:01.774 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.774 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.775 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.775 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.775 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.775 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.775 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.775 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.775 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.775 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.775 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.775 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.775 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.775 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.775 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.775 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.775 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.776 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.776 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.776 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.776 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.776 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.776 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.776 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.776 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.776 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.776 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.776 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.776 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.776 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.776 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.776 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.777 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.777 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.777 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_neorados_io 2024-03-24T12:12:01.777 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.777 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.777 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ io: /' 2024-03-24T12:12:01.777 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.777 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.777 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.777 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.777 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.777 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.777 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.777 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.777 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.777 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.778 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.778 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.778 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.778 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.778 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.778 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.778 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.778 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.778 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.778 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.778 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.778 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.778 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.778 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.779 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.779 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.779 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.779 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.779 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.779 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.779 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.779 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.779 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.779 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.779 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.779 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.779 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.779 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_neorados_io.log 2024-03-24T12:12:01.779 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.779 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.780 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82614/exe 2024-03-24T12:12:01.780 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo misc 2024-03-24T12:12:01.780 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.780 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.780 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.780 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.780 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.780 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.780 INFO:tasks.workunit.client.0.smithi006.stdout:test misc on pid 82781 2024-03-24T12:12:01.785 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=misc 2024-03-24T12:12:01.786 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82781 2024-03-24T12:12:01.786 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test misc on pid 82781' 2024-03-24T12:12:01.786 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82781 2024-03-24T12:12:01.786 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-24T12:12:01.786 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.786 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.786 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.786 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.786 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.786 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.786 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.786 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.787 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.787 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.787 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.787 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ ec_io: /' 2024-03-24T12:12:01.787 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.787 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.787 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.787 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.787 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.787 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_neorados_ec_io.log 2024-03-24T12:12:01.787 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_neorados_ec_io 2024-03-24T12:12:01.787 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.787 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.787 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.787 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.788 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.788 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.788 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_misc 2>&1 | tee ceph_test_neorados_misc.log | sed "s/^/ misc: /"' 2024-03-24T12:12:01.788 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.788 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.788 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.788 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.788 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.788 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.788 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.788 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.788 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.788 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.788 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s pool 2024-03-24T12:12:01.788 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.789 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.789 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.789 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.789 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.789 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.789 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.789 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.789 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.789 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.789 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' pool' 2024-03-24T12:12:01.789 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.789 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.789 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.789 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.790 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.790 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.790 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.790 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.790 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.790 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.790 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.790 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.790 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.790 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.790 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.790 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.790 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.790 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.790 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.791 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.791 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.791 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.791 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.791 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.791 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.791 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.791 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.791 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.791 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.791 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.791 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.791 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.791 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.791 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.792 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.792 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.792 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.792 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.792 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.792 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.792 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.792 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.792 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.792 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.792 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.792 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.792 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.792 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.792 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82685/exe 2024-03-24T12:12:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_neorados_list 2024-03-24T12:12:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.793 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.794 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.794 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.794 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.794 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.794 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.794 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.794 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.794 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.794 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.794 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.794 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.794 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.794 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.794 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.794 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.795 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.795 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.795 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.795 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.795 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.795 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.795 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.795 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.795 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.795 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.795 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.795 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.795 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.795 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.795 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo pool 2024-03-24T12:12:01.796 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82559/exe 2024-03-24T12:12:01.796 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.796 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.796 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.796 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.796 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.796 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.796 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.796 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.796 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.796 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.796 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.797 INFO:tasks.workunit.client.0.smithi006.stdout:test pool on pid 82833 2024-03-24T12:12:01.800 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=pool 2024-03-24T12:12:01.800 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82833 2024-03-24T12:12:01.800 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test pool on pid 82833' 2024-03-24T12:12:01.800 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82833 2024-03-24T12:12:01.800 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-24T12:12:01.800 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.800 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_neorados_handler_error 2024-03-24T12:12:01.800 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_neorados_handler_error.log 2024-03-24T12:12:01.801 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ handler_error: /' 2024-03-24T12:12:01.801 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.801 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.801 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.801 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.801 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.801 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.801 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.801 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.801 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.801 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.801 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.801 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.801 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.801 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.802 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.802 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.802 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.802 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.802 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.802 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.802 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_neorados_list.log 2024-03-24T12:12:01.802 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s read_operations 2024-03-24T12:12:01.802 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82535/exe 2024-03-24T12:12:01.802 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.802 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' read_operations' 2024-03-24T12:12:01.802 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_pool 2>&1 | tee ceph_test_neorados_pool.log | sed "s/^/ pool: /"' 2024-03-24T12:12:01.802 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.802 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.802 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.803 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.803 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.803 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.803 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.803 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.803 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.803 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.803 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.803 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.803 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.803 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.803 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.803 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo read_operations 2024-03-24T12:12:01.803 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.804 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ cmd: /' 2024-03-24T12:12:01.804 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.804 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.804 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.804 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.804 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.804 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.804 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.804 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.804 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.804 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.804 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.804 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.804 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.804 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.805 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.805 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.805 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.805 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.805 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_neorados_cmd 2024-03-24T12:12:01.805 INFO:tasks.workunit.client.0.smithi006.stdout:test read_operations on pid 82891 2024-03-24T12:12:01.810 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=read_operations 2024-03-24T12:12:01.811 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82891 2024-03-24T12:12:01.811 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test read_operations on pid 82891' 2024-03-24T12:12:01.811 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82891 2024-03-24T12:12:01.811 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-24T12:12:01.811 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.811 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.811 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_read_operations 2>&1 | tee ceph_test_neorados_read_operations.log | sed "s/^/ read_operations: /"' 2024-03-24T12:12:01.811 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_neorados_cmd.log 2024-03-24T12:12:01.811 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.811 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.811 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.811 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.811 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.812 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.812 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.812 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.812 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.812 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.812 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.812 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.812 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.812 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.812 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.812 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.812 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.812 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.812 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.813 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.813 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.813 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ list: /' 2024-03-24T12:12:01.813 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.813 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.813 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.813 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.813 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.813 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.813 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.813 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.813 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.813 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.813 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.813 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.814 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.814 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s snapshots 2024-03-24T12:12:01.814 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' snapshots' 2024-03-24T12:12:01.814 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.814 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.814 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.814 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.814 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.814 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.814 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.814 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.814 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.814 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.814 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.815 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.815 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.815 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.815 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.815 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.815 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.815 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.815 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.815 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.815 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.815 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.815 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.815 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.815 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.815 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.816 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.816 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.816 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.816 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.816 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.816 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.816 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.816 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.816 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.816 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.816 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.816 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.816 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.816 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.816 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.817 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.817 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.817 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.817 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.817 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.817 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.817 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.817 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.817 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.817 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.817 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.817 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.817 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.817 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.818 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.818 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.818 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.818 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.818 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.818 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.818 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.818 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.818 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.818 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.818 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.818 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.818 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo snapshots 2024-03-24T12:12:01.818 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.818 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.819 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.819 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.819 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.819 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.819 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.819 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.819 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.819 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.819 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.819 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.819 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.819 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.819 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.819 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.820 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82707/exe 2024-03-24T12:12:01.820 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.820 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.820 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.820 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.820 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.820 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.820 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.820 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.820 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.820 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.820 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.820 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.820 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.820 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.821 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.821 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.821 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.821 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.821 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.821 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.821 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.821 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.821 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.821 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.821 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.821 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.821 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.821 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.821 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.822 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.822 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.822 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.822 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.822 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.822 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.822 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.822 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.822 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.822 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.822 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.822 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.822 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.822 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=snapshots 2024-03-24T12:12:01.822 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82975 2024-03-24T12:12:01.823 INFO:tasks.workunit.client.0.smithi006.stdout:test snapshots on pid 82975 2024-03-24T12:12:01.825 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test snapshots on pid 82975' 2024-03-24T12:12:01.825 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82975 2024-03-24T12:12:01.825 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-24T12:12:01.825 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.825 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_snapshots 2>&1 | tee ceph_test_neorados_snapshots.log | sed "s/^/ snapshots: /"' 2024-03-24T12:12:01.825 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s watch_notify 2024-03-24T12:12:01.826 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.826 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.826 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.826 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.826 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.826 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_neorados_ec_list 2024-03-24T12:12:01.826 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.826 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.826 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.826 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' watch_notify' 2024-03-24T12:12:01.826 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.826 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.826 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.826 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.827 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.827 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.827 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.827 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.827 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.827 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_neorados_ec_list.log 2024-03-24T12:12:01.827 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.827 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.827 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.827 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.827 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.827 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.827 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.827 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.827 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.828 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.828 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.828 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.828 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.828 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.828 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.828 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.828 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.828 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.828 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ ec_list: /' 2024-03-24T12:12:01.828 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.828 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.828 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.828 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.828 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.829 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.829 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.829 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.829 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.829 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.829 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.829 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.829 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.829 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.829 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.829 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.829 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.829 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.829 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.829 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.830 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.830 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.830 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.830 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo watch_notify 2024-03-24T12:12:01.830 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.830 INFO:tasks.workunit.client.0.smithi006.stdout:test watch_notify on pid 82989 2024-03-24T12:12:01.835 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=watch_notify 2024-03-24T12:12:01.835 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82989 2024-03-24T12:12:01.835 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test watch_notify on pid 82989' 2024-03-24T12:12:01.835 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=82989 2024-03-24T12:12:01.836 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-24T12:12:01.836 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.836 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.836 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.836 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.836 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.836 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.836 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.836 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.836 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.836 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.836 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.836 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.836 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.837 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.837 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.837 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.837 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.837 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.837 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.837 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.837 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.837 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_watch_notify 2>&1 | tee ceph_test_neorados_watch_notify.log | sed "s/^/ watch_notify: /"' 2024-03-24T12:12:01.837 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.837 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.837 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.837 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.837 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.837 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.838 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.838 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.838 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.838 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.838 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.838 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.838 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.838 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.838 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.838 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.838 INFO:tasks.workunit.client.0.smithi006.stderr:++ printf %25s write_operations 2024-03-24T12:12:01.838 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.838 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.838 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.839 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.839 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.839 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.839 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.839 INFO:tasks.workunit.client.0.smithi006.stderr:+ r=' write_operations' 2024-03-24T12:12:01.839 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.839 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.839 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.839 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.839 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.839 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.839 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.839 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.839 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.839 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.840 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.840 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.840 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.840 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.840 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.840 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.840 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.840 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.840 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.840 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.840 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.840 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.840 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.840 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.840 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.841 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.841 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.841 INFO:tasks.workunit.client.0.smithi006.stderr:++ awk '{print $1}' 2024-03-24T12:12:01.841 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82781/exe 2024-03-24T12:12:01.841 INFO:tasks.workunit.client.0.smithi006.stderr:++ echo write_operations 2024-03-24T12:12:01.841 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.841 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.841 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.841 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.841 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.841 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.841 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.841 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.841 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.841 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.842 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.842 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.842 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.842 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.842 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.842 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.842 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.842 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.842 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.842 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.842 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.842 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.842 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.842 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.842 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.843 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.843 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.843 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.843 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.843 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.843 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.843 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.843 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.843 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.843 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.843 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.843 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.843 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.843 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.843 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.844 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.844 INFO:tasks.workunit.client.0.smithi006.stdout:test write_operations on pid 83007 2024-03-24T12:12:01.855 INFO:tasks.workunit.client.0.smithi006.stderr:+ ff=write_operations 2024-03-24T12:12:01.855 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=83007 2024-03-24T12:12:01.855 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo 'test write_operations on pid 83007' 2024-03-24T12:12:01.856 INFO:tasks.workunit.client.0.smithi006.stderr:+ pids[$f]=83007 2024-03-24T12:12:01.856 INFO:tasks.workunit.client.0.smithi006.stderr:+ ret=0 2024-03-24T12:12:01.856 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.856 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.856 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 1 -eq 1 ']' 2024-03-24T12:12:01.856 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.856 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.856 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:12:01.856 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81873 2024-03-24T12:12:01.857 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.857 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 81873 2024-03-24T12:12:01.857 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.857 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.857 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82891/exe 2024-03-24T12:12:01.857 INFO:tasks.workunit.client.0.smithi006.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_write_operations 2>&1 | tee ceph_test_neorados_write_operations.log | sed "s/^/ write_operations: /"' 2024-03-24T12:12:01.857 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.857 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.857 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.857 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_neorados_misc.log 2024-03-24T12:12:01.857 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.858 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.858 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.858 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.858 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.858 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.858 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.858 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.858 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.858 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ misc: /' 2024-03-24T12:12:01.858 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_neorados_misc 2024-03-24T12:12:01.858 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -f /etc/bashrc ']' 2024-03-24T12:12:01.858 INFO:tasks.workunit.client.0.smithi006.stderr:+ . /etc/bashrc 2024-03-24T12:12:01.858 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.858 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -z '' ']' 2024-03-24T12:12:01.859 INFO:tasks.workunit.client.0.smithi006.stderr:++ BASHRCSOURCED=Y 2024-03-24T12:12:01.859 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.859 INFO:tasks.workunit.client.0.smithi006.stderr:++ shopt -q login_shell 2024-03-24T12:12:01.859 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.859 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.859 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.859 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.859 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.859 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.859 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.859 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.859 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.859 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.860 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.860 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.860 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.860 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.860 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.860 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.860 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.860 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.860 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.860 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.860 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.860 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.861 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.861 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.861 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.861 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.861 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.861 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.861 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.861 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.861 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.861 INFO:tasks.workunit.client.0.smithi006.stderr:+++ umask 2024-03-24T12:12:01.861 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_neorados_read_operations 2024-03-24T12:12:01.861 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.861 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.861 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.861 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.862 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.862 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' 0022 -eq 0 ']' 2024-03-24T12:12:01.862 INFO:tasks.workunit.client.0.smithi006.stderr:++ SHELL=/bin/bash 2024-03-24T12:12:01.862 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.862 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-24T12:12:01.862 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.862 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-24T12:12:01.862 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.862 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-24T12:12:01.862 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.862 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-24T12:12:01.862 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.862 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-24T12:12:01.862 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.863 INFO:tasks.workunit.client.0.smithi006.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-24T12:12:01.863 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.863 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-24T12:12:01.863 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.863 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-24T12:12:01.863 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.863 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ read_operations: /' 2024-03-24T12:12:01.863 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_neorados_read_operations.log 2024-03-24T12:12:01.863 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.863 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.863 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.863 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.863 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.863 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.863 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.864 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.864 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.864 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.864 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.864 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.864 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.864 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.864 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.864 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.864 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.864 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.864 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.864 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.864 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.864 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.865 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.865 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.865 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.865 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.865 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.865 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.865 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.865 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.865 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.865 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.865 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.866 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82833/exe 2024-03-24T12:12:01.866 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.866 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.866 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.866 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.866 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.866 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.866 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.866 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'grep=grep --color=auto' 2024-03-24T12:12:01.867 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-24T12:12:01.867 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-24T12:12:01.867 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.867 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-24T12:12:01.867 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.867 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorls.sh 2024-03-24T12:12:01.867 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' '!' -t 0 ']' 2024-03-24T12:12:01.867 INFO:tasks.workunit.client.0.smithi006.stderr:+++ return 2024-03-24T12:12:01.867 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.867 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-24T12:12:01.867 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.868 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-24T12:12:01.868 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export S_COLORS=auto 2024-03-24T12:12:01.868 INFO:tasks.workunit.client.0.smithi006.stderr:+++ S_COLORS=auto 2024-03-24T12:12:01.868 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.868 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-24T12:12:01.868 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.868 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-24T12:12:01.868 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.868 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.868 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.868 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.868 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.868 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.868 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.868 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.869 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.869 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.869 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.869 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.869 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.869 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.869 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.869 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.869 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.869 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.869 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.869 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.869 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.869 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.869 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.869 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.870 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.870 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_neorados_pool 2024-03-24T12:12:01.870 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.870 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.870 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.870 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.870 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.870 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.870 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.870 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.870 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_neorados_pool.log 2024-03-24T12:12:01.870 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.870 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.870 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.871 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.871 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.871 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.871 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.871 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.871 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.871 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.871 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.871 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-24T12:12:01.871 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-24T12:12:01.871 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-24T12:12:01.871 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.871 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-24T12:12:01.871 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.871 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-24T12:12:01.871 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-24T12:12:01.872 INFO:tasks.workunit.client.0.smithi006.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-24T12:12:01.872 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.872 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.872 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.872 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.872 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.872 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.872 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.872 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.872 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.872 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.872 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.872 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.872 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.872 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.873 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.873 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.873 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.873 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.873 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.873 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.873 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.873 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.873 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.873 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.873 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.873 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.873 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ pool: /' 2024-03-24T12:12:01.873 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82975/exe 2024-03-24T12:12:01.873 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.873 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.874 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.874 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.874 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.874 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.874 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.874 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.874 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.874 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.874 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.874 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_neorados_snapshots 2024-03-24T12:12:01.874 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ snapshots: /' 2024-03-24T12:12:01.874 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-24T12:12:01.874 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-24T12:12:01.875 INFO:tasks.workunit.client.0.smithi006.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-24T12:12:01.875 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.875 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-24T12:12:01.875 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.875 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-24T12:12:01.875 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-24T12:12:01.875 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.875 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-24T12:12:01.875 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.875 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/gawk.sh 2024-03-24T12:12:01.875 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.875 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-24T12:12:01.875 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.875 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/lang.sh 2024-03-24T12:12:01.875 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup 2024-03-24T12:12:01.875 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/locale 2024-03-24T12:12:01.876 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.876 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_neorados_snapshots.log 2024-03-24T12:12:01.876 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.876 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-24T12:12:01.876 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.876 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-24T12:12:01.876 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-24T12:12:01.876 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.876 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.876 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.876 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.876 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.876 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.876 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.877 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.877 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.877 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.877 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.877 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.877 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.877 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.877 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.877 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.877 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.877 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.877 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.877 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.877 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.877 INFO:tasks.workunit.client.0.smithi006.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-24T12:12:01.878 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/82989/exe 2024-03-24T12:12:01.878 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.878 INFO:tasks.workunit.client.0.smithi006.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-24T12:12:01.878 INFO:tasks.workunit.client.0.smithi006.stderr:++++ export LANG=en_US.UTF-8 2024-03-24T12:12:01.878 INFO:tasks.workunit.client.0.smithi006.stderr:++++ LANG=en_US.UTF-8 2024-03-24T12:12:01.878 INFO:tasks.workunit.client.0.smithi006.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-24T12:12:01.878 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-24T12:12:01.878 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.878 INFO:tasks.workunit.client.0.smithi006.stderr:+++ LANG=en_US.UTF-8 2024-03-24T12:12:01.878 INFO:tasks.workunit.client.0.smithi006.stderr:+++ unset LANG_backup config 2024-03-24T12:12:01.878 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n '' ']' 2024-03-24T12:12:01.878 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-24T12:12:01.878 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' dumb = linux ']' 2024-03-24T12:12:01.878 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.878 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-24T12:12:01.879 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.879 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/less.sh 2024-03-24T12:12:01.879 INFO:tasks.workunit.client.0.smithi006.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-24T12:12:01.879 INFO:tasks.workunit.client.0.smithi006.stderr:++ for i in /etc/profile.d/*.sh 2024-03-24T12:12:01.879 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-24T12:12:01.879 INFO:tasks.workunit.client.0.smithi006.stderr:++ '[' '' ']' 2024-03-24T12:12:01.879 INFO:tasks.workunit.client.0.smithi006.stderr:++ . /etc/profile.d/which2.sh 2024-03-24T12:12:01.879 INFO:tasks.workunit.client.0.smithi006.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-24T12:12:01.879 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.879 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.879 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.879 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.879 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.879 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.879 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.880 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.880 INFO:tasks.workunit.client.0.smithi006.stderr:+++++ readlink /proc/83007/exe 2024-03-24T12:12:01.880 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.880 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.880 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ watch_notify: /' 2024-03-24T12:12:01.880 INFO:tasks.workunit.client.0.smithi006.stderr:++++ basename /usr/bin/bash 2024-03-24T12:12:01.880 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_neorados_watch_notify.log 2024-03-24T12:12:01.880 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_declare='declare -f' 2024-03-24T12:12:01.880 INFO:tasks.workunit.client.0.smithi006.stderr:+++ which_opt=-f 2024-03-24T12:12:01.884 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export which_declare 2024-03-24T12:12:01.884 INFO:tasks.workunit.client.0.smithi006.stderr:+++ export -f which 2024-03-24T12:12:01.885 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset i 2024-03-24T12:12:01.885 INFO:tasks.workunit.client.0.smithi006.stderr:++ unset -f pathmunge 2024-03-24T12:12:01.885 INFO:tasks.workunit.client.0.smithi006.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-24T12:12:01.885 INFO:tasks.workunit.client.0.smithi006.stderr:+ export PATH 2024-03-24T12:12:01.885 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-24T12:12:01.885 INFO:tasks.workunit.client.0.smithi006.stderr:+ unset rc 2024-03-24T12:12:01.885 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_neorados_write_operations 2024-03-24T12:12:01.885 INFO:tasks.workunit.client.0.smithi006.stderr:+ sed 's/^/ write_operations: /' 2024-03-24T12:12:01.885 INFO:tasks.workunit.client.0.smithi006.stderr:+ tee ceph_test_neorados_write_operations.log 2024-03-24T12:12:01.885 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph_test_neorados_watch_notify 2024-03-24T12:12:02.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:12:02 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:12:02.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:02 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:12:02.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:02 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1149679251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:02.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:02 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2724215036' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-03-24T12:12:02.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:02 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3070372572' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi006-82972-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:02 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:12:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:02 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1149679251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:02 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2724215036' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-03-24T12:12:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:02 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3070372572' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi006-82972-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:02.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:02 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:12:02.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:02 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1149679251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:02.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:02 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2724215036' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-03-24T12:12:02.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:02 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3070372572' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi006-82972-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3068853266' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi006-81794-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2378966145' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi006-81943-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4267194880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi006-81813-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2607029818' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi006-81775-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:03 smithi090 ceph-mon[33646]: osdmap e73: 8 total, 8 up, 8 in 2024-03-24T12:12:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3068853266' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi006-81794-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2378966145' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi006-81943-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4267194880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi006-81813-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2607029818' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi006-81775-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:03 smithi006 ceph-mon[30108]: osdmap e73: 8 total, 8 up, 8 in 2024-03-24T12:12:03.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3068853266' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi006-81794-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:03.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2378966145' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi006-81943-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:03.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4267194880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi006-81813-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:03.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2607029818' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi006-81775-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:03.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:03 smithi006 ceph-mon[41734]: osdmap e73: 8 total, 8 up, 8 in 2024-03-24T12:12:04.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:04 smithi090 ceph-mon[33646]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:12:04.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:04 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4125754551' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi006-82140-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:04.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:04 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1070475865' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi006-82076-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:04.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:04 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1843173508' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi006-81895-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:04.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:04 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi006-81868-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:04.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:04 smithi090 ceph-mon[33646]: osdmap e74: 8 total, 8 up, 8 in 2024-03-24T12:12:04.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:04 smithi006 ceph-mon[30108]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:12:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:04 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4125754551' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi006-82140-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:04 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1070475865' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi006-82076-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:04 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1843173508' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi006-81895-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:04 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi006-81868-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:04 smithi006 ceph-mon[30108]: osdmap e74: 8 total, 8 up, 8 in 2024-03-24T12:12:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:04 smithi006 ceph-mon[41734]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:12:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:04 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4125754551' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi006-82140-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:04 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1070475865' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi006-82076-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:04 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1843173508' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi006-81895-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:04 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi006-81868-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:04 smithi006 ceph-mon[41734]: osdmap e74: 8 total, 8 up, 8 in 2024-03-24T12:12:05.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:12:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:12:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:12:10.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:09 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:12:10.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:09 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi006-82783-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:10.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:09 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:10.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:09 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi006-82972-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:10.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:09 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi006-81794-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:10.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:09 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi006-81943-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:10.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:09 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:12:10.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:09 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi006-82783-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:10.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:09 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:10.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:09 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi006-82972-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:10.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:09 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi006-81794-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:10.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:09 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi006-81943-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:10.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:09 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:12:10.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:09 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi006-82783-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:10.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:09 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:10.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:09 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi006-82972-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:10.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:09 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi006-81794-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:10.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:09 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi006-81943-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:10 smithi090 ceph-mon[33646]: pgmap v65: 260 pgs: 128 unknown, 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:12:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1893647500' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1093636271' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi006-82159-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1149679251' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:10 smithi090 ceph-mon[33646]: osdmap e75: 8 total, 8 up, 8 in 2024-03-24T12:12:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:10 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi006-81813-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:10 smithi090 ceph-mon[33646]: Health check failed: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:11.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:10 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi006-82140-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:10 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi006-82076-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:10 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi006-81775-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3036896088' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3951271448' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82288-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:10 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi006-81895-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[30108]: pgmap v65: 260 pgs: 128 unknown, 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:12:11.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1893647500' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1093636271' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi006-82159-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1149679251' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:11.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[30108]: osdmap e75: 8 total, 8 up, 8 in 2024-03-24T12:12:11.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi006-81813-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[30108]: Health check failed: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:11.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi006-82140-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi006-82076-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi006-81775-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3036896088' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3951271448' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82288-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi006-81895-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[41734]: pgmap v65: 260 pgs: 128 unknown, 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:12:11.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1893647500' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1093636271' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi006-82159-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1149679251' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:11.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[41734]: osdmap e75: 8 total, 8 up, 8 in 2024-03-24T12:12:11.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi006-81813-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[41734]: Health check failed: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:11.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi006-82140-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi006-82076-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi006-81775-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3036896088' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3951271448' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82288-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:11.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:10 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi006-81895-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.130 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: pgmap v68: 516 pgs: 384 unknown, 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:12:12.130 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: pgmap v69: 516 pgs: 250 creating+peering, 194 active+clean, 72 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-24T12:12:12.130 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: pgmap v70: 516 pgs: 250 creating+peering, 194 active+clean, 72 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 785 B/s rd, 0 op/s 2024-03-24T12:12:12.130 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi006-82783-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:12.130 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi006-82972-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:12.130 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi006-81794-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.130 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi006-81943-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.130 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2243327668' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi006-82137-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.130 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1483124200' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi006-82543-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.131 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/569555403' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi006-82061-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.131 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2344039957' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi006-82180-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.131 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.131 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1510313285' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82516-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.131 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3070372572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi006-82972-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi006-82972-1"}]: dispatch 2024-03-24T12:12:12.131 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/310149184' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi006-81832-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.131 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: osdmap e76: 8 total, 8 up, 8 in 2024-03-24T12:12:12.131 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi006-81868-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.131 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi006-82783-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi006-82783-1"}]: dispatch 2024-03-24T12:12:12.131 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82516-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.131 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi006-82972-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi006-82972-1"}]: dispatch 2024-03-24T12:12:12.131 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.131 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi006-81832-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.131 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi006-82159-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.132 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:12.132 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi006-82140-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.132 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi006-82076-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.132 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi006-81813-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.132 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi006-81775-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.132 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3036896088' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.132 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3951271448' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82288-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.132 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi006-81895-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.132 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi006-81868-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.132 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: osdmap e77: 8 total, 8 up, 8 in 2024-03-24T12:12:12.132 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi006-82137-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.132 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1149679251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:12.132 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi006-82543-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.132 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi006-82061-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.133 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi006-82180-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.133 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.133 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1942235919' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82421-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.133 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:12.133 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1455428284' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-24T12:12:12.133 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82421-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.133 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:11 smithi090 ceph-mon[33646]: pgmap v73: 1284 pgs: 250 creating+peering, 194 active+clean, 840 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 794 B/s rd, 0 op/s 2024-03-24T12:12:12.196 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-03-24T12:12:12.196 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [----------] Global test environment set-up. 2024-03-24T12:12:12.196 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [----------] 12 tests from AsioRados 2024-03-24T12:12:12.196 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-03-24T12:12:12.196 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (1 ms) 2024-03-24T12:12:12.197 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-03-24T12:12:12.197 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (1 ms) 2024-03-24T12:12:12.197 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-03-24T12:12:12.197 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (0 ms) 2024-03-24T12:12:12.197 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-03-24T12:12:12.197 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (40 ms) 2024-03-24T12:12:12.197 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-03-24T12:12:12.197 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (4 ms) 2024-03-24T12:12:12.197 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-03-24T12:12:12.197 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (4 ms) 2024-03-24T12:12:12.197 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-03-24T12:12:12.197 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (1 ms) 2024-03-24T12:12:12.197 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-03-24T12:12:12.197 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (0 ms) 2024-03-24T12:12:12.197 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-03-24T12:12:12.198 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (0 ms) 2024-03-24T12:12:12.198 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-03-24T12:12:12.198 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (5 ms) 2024-03-24T12:12:12.198 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-03-24T12:12:12.198 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (2 ms) 2024-03-24T12:12:12.198 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-03-24T12:12:12.198 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (3 ms) 2024-03-24T12:12:12.198 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [----------] 12 tests from AsioRados (62 ms total) 2024-03-24T12:12:12.198 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: 2024-03-24T12:12:12.198 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [----------] Global test environment tear-down 2024-03-24T12:12:12.198 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (10786 ms total) 2024-03-24T12:12:12.198 INFO:tasks.workunit.client.0.smithi006.stdout: api_asio: [ PASSED ] 12 tests. 2024-03-24T12:12:12.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: pgmap v68: 516 pgs: 384 unknown, 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:12:12.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: pgmap v69: 516 pgs: 250 creating+peering, 194 active+clean, 72 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-24T12:12:12.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: pgmap v70: 516 pgs: 250 creating+peering, 194 active+clean, 72 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 785 B/s rd, 0 op/s 2024-03-24T12:12:12.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi006-82783-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:12.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi006-82972-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:12.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi006-81794-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi006-81943-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2243327668' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi006-82137-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1483124200' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi006-82543-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/569555403' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi006-82061-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2344039957' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi006-82180-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1510313285' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82516-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3070372572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi006-82972-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi006-82972-1"}]: dispatch 2024-03-24T12:12:12.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/310149184' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi006-81832-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: osdmap e76: 8 total, 8 up, 8 in 2024-03-24T12:12:12.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi006-81868-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: pgmap v68: 516 pgs: 384 unknown, 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:12:12.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: pgmap v69: 516 pgs: 250 creating+peering, 194 active+clean, 72 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-24T12:12:12.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: pgmap v70: 516 pgs: 250 creating+peering, 194 active+clean, 72 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 785 B/s rd, 0 op/s 2024-03-24T12:12:12.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi006-82783-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:12.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi006-82972-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:12.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi006-81794-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi006-81943-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2243327668' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi006-82137-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1483124200' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi006-82543-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/569555403' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi006-82061-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2344039957' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi006-82180-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1510313285' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82516-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3070372572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi006-82972-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi006-82972-1"}]: dispatch 2024-03-24T12:12:12.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/310149184' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi006-81832-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: osdmap e76: 8 total, 8 up, 8 in 2024-03-24T12:12:12.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi006-81868-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.214 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi006-82783-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi006-82783-1"}]: dispatch 2024-03-24T12:12:12.214 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82516-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.214 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi006-82972-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi006-82972-1"}]: dispatch 2024-03-24T12:12:12.214 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.214 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi006-81832-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.214 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi006-82159-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.214 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:12.214 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi006-82140-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.214 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi006-82076-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi006-81813-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi006-81775-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3036896088' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3951271448' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82288-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi006-81895-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi006-81868-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: osdmap e77: 8 total, 8 up, 8 in 2024-03-24T12:12:12.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi006-82137-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1149679251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:12.215 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi006-82783-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi006-82783-1"}]: dispatch 2024-03-24T12:12:12.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82516-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi006-82972-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi006-82972-1"}]: dispatch 2024-03-24T12:12:12.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi006-81832-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi006-82159-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:12.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi006-82140-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi006-82076-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi006-81813-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi006-81775-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3036896088' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3951271448' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82288-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi006-81895-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi006-81868-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:12.217 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: osdmap e77: 8 total, 8 up, 8 in 2024-03-24T12:12:12.217 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi006-82137-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.217 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1149679251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:12.217 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi006-82543-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.217 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi006-82061-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.217 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi006-82180-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.217 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.217 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1942235919' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82421-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.217 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:12.217 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1455428284' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-24T12:12:12.217 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82421-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.217 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[41734]: pgmap v73: 1284 pgs: 250 creating+peering, 194 active+clean, 840 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 794 B/s rd, 0 op/s 2024-03-24T12:12:12.217 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi006-82543-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.218 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi006-82061-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.218 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi006-82180-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.218 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.218 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1942235919' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82421-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.218 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:12.218 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1455428284' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-24T12:12:12.218 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82421-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:12.218 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:11 smithi006 ceph-mon[30108]: pgmap v73: 1284 pgs: 250 creating+peering, 194 active+clean, 840 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 794 B/s rd, 0 op/s 2024-03-24T12:12:12.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:12:12 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:12:12.441 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-03-24T12:12:12.441 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [----------] Global test environment set-up. 2024-03-24T12:12:12.441 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [----------] 7 tests from LibRadosList 2024-03-24T12:12:12.442 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-03-24T12:12:12.442 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [ OK ] LibRadosList.ListObjects (1341 ms) 2024-03-24T12:12:12.442 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-03-24T12:12:12.442 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (54 ms) 2024-03-24T12:12:12.442 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-03-24T12:12:12.442 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: myset foo1,foo2,foo3 2024-03-24T12:12:12.442 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: foo1 2024-03-24T12:12:12.442 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: foo2 2024-03-24T12:12:12.442 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: foo3 2024-03-24T12:12:12.442 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: myset foo1,foo4,foo5 2024-03-24T12:12:12.442 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: foo4 2024-03-24T12:12:12.442 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: foo5 2024-03-24T12:12:12.442 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: foo1 2024-03-24T12:12:12.442 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: myset foo6,foo7 2024-03-24T12:12:12.442 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: foo7 2024-03-24T12:12:12.443 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: foo6 2024-03-24T12:12:12.443 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-03-24T12:12:12.443 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: ns1:foo4 2024-03-24T12:12:12.443 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: ns1:foo5 2024-03-24T12:12:12.443 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: ns2:foo7 2024-03-24T12:12:12.443 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: ns2:foo6 2024-03-24T12:12:12.443 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: ns1:foo1 2024-03-24T12:12:12.443 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: :foo1 2024-03-24T12:12:12.443 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: :foo2 2024-03-24T12:12:12.443 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: :foo3 2024-03-24T12:12:12.443 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (90 ms) 2024-03-24T12:12:12.443 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-03-24T12:12:12.443 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 1 0 2024-03-24T12:12:12.443 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 10 0 2024-03-24T12:12:12.443 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 13 0 2024-03-24T12:12:12.444 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 7 0 2024-03-24T12:12:12.444 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 14 0 2024-03-24T12:12:12.444 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 0 0 2024-03-24T12:12:12.444 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 15 0 2024-03-24T12:12:12.444 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 11 0 2024-03-24T12:12:12.444 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 5 0 2024-03-24T12:12:12.444 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 8 0 2024-03-24T12:12:12.444 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 6 0 2024-03-24T12:12:12.444 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 3 0 2024-03-24T12:12:12.444 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 4 0 2024-03-24T12:12:12.444 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 12 0 2024-03-24T12:12:12.444 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 9 0 2024-03-24T12:12:12.444 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 2 0 2024-03-24T12:12:12.444 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-03-24T12:12:12.444 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (68 ms) 2024-03-24T12:12:12.444 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-03-24T12:12:12.445 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: x cursor=MIN 2024-03-24T12:12:12.445 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > oid=1 cursor=11:02547ec2:::1:head 2024-03-24T12:12:12.445 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > oid=10 cursor=11:52ea6a34:::10:head 2024-03-24T12:12:12.445 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > oid=13 cursor=11:566253c9:::13:head 2024-03-24T12:12:12.445 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > oid=7 cursor=11:5c6b0b28:::7:head 2024-03-24T12:12:12.445 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > oid=14 cursor=11:62a1935d:::14:head 2024-03-24T12:12:12.445 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > oid=0 cursor=11:6cac518f:::0:head 2024-03-24T12:12:12.445 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > oid=15 cursor=11:863748b0:::15:head 2024-03-24T12:12:12.445 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > oid=11 cursor=11:89d3ae78:::11:head 2024-03-24T12:12:12.445 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > oid=5 cursor=11:b29083e3:::5:head 2024-03-24T12:12:12.445 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > oid=8 cursor=11:bd63b0f1:::8:head 2024-03-24T12:12:12.445 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > oid=6 cursor=11:c4fdafeb:::6:head 2024-03-24T12:12:12.445 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > oid=3 cursor=11:cfc208b3:::3:head 2024-03-24T12:12:12.445 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > oid=4 cursor=11:d83876eb:::4:head 2024-03-24T12:12:12.445 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > oid=12 cursor=11:de5d7c5f:::12:head 2024-03-24T12:12:12.445 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > oid=9 cursor=11:e960b815:::9:head 2024-03-24T12:12:12.446 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > oid=2 cursor=11:f905c69b:::2:head 2024-03-24T12:12:12.446 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: FIRST> seek to MIN oid=1 2024-03-24T12:12:12.446 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : oid=1 cursor=11:02547ec2:::1:head 2024-03-24T12:12:12.446 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:02547ec2:::1:head 2024-03-24T12:12:12.446 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:02547ec2:::1:head -> 1 2024-03-24T12:12:12.446 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : oid=10 cursor=11:52ea6a34:::10:head 2024-03-24T12:12:12.446 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:52ea6a34:::10:head 2024-03-24T12:12:12.446 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:52ea6a34:::10:head -> 10 2024-03-24T12:12:12.446 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : oid=13 cursor=11:566253c9:::13:head 2024-03-24T12:12:12.446 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:566253c9:::13:head 2024-03-24T12:12:12.446 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:566253c9:::13:head -> 13 2024-03-24T12:12:12.446 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : oid=7 cursor=11:5c6b0b28:::7:head 2024-03-24T12:12:12.446 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:5c6b0b28:::7:head 2024-03-24T12:12:12.446 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:5c6b0b28:::7:head -> 7 2024-03-24T12:12:12.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : oid=14 cursor=11:62a1935d:::14:head 2024-03-24T12:12:12.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:62a1935d:::14:head 2024-03-24T12:12:12.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:62a1935d:::14:head -> 14 2024-03-24T12:12:12.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : oid=0 cursor=11:6cac518f:::0:head 2024-03-24T12:12:12.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:6cac518f:::0:head 2024-03-24T12:12:12.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:6cac518f:::0:head -> 0 2024-03-24T12:12:12.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : oid=15 cursor=11:863748b0:::15:head 2024-03-24T12:12:12.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:863748b0:::15:head 2024-03-24T12:12:12.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:863748b0:::15:head -> 15 2024-03-24T12:12:12.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : oid=11 cursor=11:89d3ae78:::11:head 2024-03-24T12:12:12.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:89d3ae78:::11:head 2024-03-24T12:12:12.548 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:89d3ae78:::11:head -> 11 2024-03-24T12:12:12.548 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : oid=5 cursor=11:b29083e3:::5:head 2024-03-24T12:12:12.548 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:b29083e3:::5:head 2024-03-24T12:12:12.548 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:b29083e3:::5:head -> 5 2024-03-24T12:12:12.548 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : oid=8 cursor=11:bd63b0f1:::8:head 2024-03-24T12:12:12.548 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:bd63b0f1:::8:head 2024-03-24T12:12:12.548 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:bd63b0f1:::8:head -> 8 2024-03-24T12:12:12.548 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : oid=6 cursor=11:c4fdafeb:::6:head 2024-03-24T12:12:12.548 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:c4fdafeb:::6:head 2024-03-24T12:12:12.548 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:c4fdafeb:::6:head -> 6 2024-03-24T12:12:12.548 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : oid=3 cursor=11:cfc208b3:::3:head 2024-03-24T12:12:12.548 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:cfc208b3:::3:head 2024-03-24T12:12:12.548 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:cfc208b3:::3:head -> 3 2024-03-24T12:12:12.548 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : oid=4 cursor=11:d83876eb:::4:head 2024-03-24T12:12:12.548 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:d83876eb:::4:head 2024-03-24T12:12:12.548 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:d83876eb:::4:head -> 4 2024-03-24T12:12:12.549 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : oid=12 cursor=11:de5d7c5f:::12:head 2024-03-24T12:12:12.549 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:de5d7c5f:::12:head 2024-03-24T12:12:12.549 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:de5d7c5f:::12:head -> 12 2024-03-24T12:12:12.549 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : oid=9 cursor=11:e960b815:::9:head 2024-03-24T12:12:12.549 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:e960b815:::9:head 2024-03-24T12:12:12.549 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:e960b815:::9:head -> 9 2024-03-24T12:12:12.549 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : oid=2 cursor=11:f905c69b:::2:head 2024-03-24T12:12:12.549 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:f905c69b:::2:head 2024-03-24T12:12:12.549 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:f905c69b:::2:head -> 2 2024-03-24T12:12:12.549 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:52ea6a34:::10:head 2024-03-24T12:12:12.549 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : cursor()=11:52ea6a34:::10:head expected=11:52ea6a34:::10:head 2024-03-24T12:12:12.549 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:52ea6a34:::10:head -> 10 2024-03-24T12:12:12.549 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : entry=10 expected=10 2024-03-24T12:12:12.549 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:f905c69b:::2:head 2024-03-24T12:12:12.550 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : cursor()=11:f905c69b:::2:head expected=11:f905c69b:::2:head 2024-03-24T12:12:12.550 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:f905c69b:::2:head -> 2 2024-03-24T12:12:12.550 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : entry=2 expected=2 2024-03-24T12:12:12.550 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:cfc208b3:::3:head 2024-03-24T12:12:12.550 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : cursor()=11:cfc208b3:::3:head expected=11:cfc208b3:::3:head 2024-03-24T12:12:12.550 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:cfc208b3:::3:head -> 3 2024-03-24T12:12:12.550 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : entry=3 expected=3 2024-03-24T12:12:12.550 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:c4fdafeb:::6:head 2024-03-24T12:12:12.550 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : cursor()=11:c4fdafeb:::6:head expected=11:c4fdafeb:::6:head 2024-03-24T12:12:12.550 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:c4fdafeb:::6:head -> 6 2024-03-24T12:12:12.550 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : entry=6 expected=6 2024-03-24T12:12:12.550 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:bd63b0f1:::8:head 2024-03-24T12:12:12.550 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : cursor()=11:bd63b0f1:::8:head expected=11:bd63b0f1:::8:head 2024-03-24T12:12:12.550 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:bd63b0f1:::8:head -> 8 2024-03-24T12:12:12.550 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : entry=8 expected=8 2024-03-24T12:12:12.551 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:b29083e3:::5:head 2024-03-24T12:12:12.551 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : cursor()=11:b29083e3:::5:head expected=11:b29083e3:::5:head 2024-03-24T12:12:12.551 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:b29083e3:::5:head -> 5 2024-03-24T12:12:12.551 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : entry=5 expected=5 2024-03-24T12:12:12.551 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:89d3ae78:::11:head 2024-03-24T12:12:12.551 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : cursor()=11:89d3ae78:::11:head expected=11:89d3ae78:::11:head 2024-03-24T12:12:12.551 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:89d3ae78:::11:head -> 11 2024-03-24T12:12:12.551 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : entry=11 expected=11 2024-03-24T12:12:12.551 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:02547ec2:::1:head 2024-03-24T12:12:12.551 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : cursor()=11:02547ec2:::1:head expected=11:02547ec2:::1:head 2024-03-24T12:12:12.551 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:02547ec2:::1:head -> 1 2024-03-24T12:12:12.815 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : entry=1 list_parallel: process_1_[82576]: starting. 2024-03-24T12:12:12.815 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_1_[82576]: creating pool ceph_test_rados_list_parallel.smithi006-82538 2024-03-24T12:12:12.815 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_1_[82576]: created object 0... 2024-03-24T12:12:12.815 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_1_[82576]: created object 25... 2024-03-24T12:12:12.815 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_1_[82576]: created object 49... 2024-03-24T12:12:12.815 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_1_[82576]: finishing. 2024-03-24T12:12:12.815 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_1_[82576]: shutting down. 2024-03-24T12:12:12.815 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_2_[82577]: starting. 2024-03-24T12:12:12.816 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_2_[82577]: listing objects. 2024-03-24T12:12:12.816 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_2_[82577]: listed object 0... 2024-03-24T12:12:12.816 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_2_[82577]: listed object 25... 2024-03-24T12:12:12.816 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_2_[82577]: saw 50 objects 2024-03-24T12:12:12.816 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_2_[82577]: shutting down. 2024-03-24T12:12:12.816 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:12.816 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_3_[83731]: starting. 2024-03-24T12:12:12.816 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_3_[83731]: creating pool ceph_test_rados_list_parallel.smithi006-82538 2024-03-24T12:12:12.816 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_3_[83731]: created object 0... 2024-03-24T12:12:12.816 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_3_[83731]: created object 25... 2024-03-24T12:12:12.816 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_3_[83731]: created object 49... 2024-03-24T12:12:12.816 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_3_[83731]: finishing. 2024-03-24T12:12:12.816 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_3_[83731]: shutting down. 2024-03-24T12:12:12.816 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:12.816 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_4_[83732]: starting. 2024-03-24T12:12:12.817 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_4_[83732]: listing objects. 2024-03-24T12:12:12.817 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_4_[83732]: listed object 0... 2024-03-24T12:12:12.817 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_4_[83732]: listed object 25... 2024-03-24T12:12:12.817 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_4_[83732]: saw 45 objects 2024-03-24T12:12:12.817 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_4_[83732]: shutting down. 2024-03-24T12:12:12.817 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:12.817 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_5_[83733]: starting. 2024-03-24T12:12:12.817 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_5_[83733]: removed 25 objects... 2024-03-24T12:12:12.817 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_5_[83733]: removed half of the objects 2024-03-24T12:12:12.817 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_5_[83733]: removed 50 objects... 2024-03-24T12:12:12.817 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_5_[83733]: removed 50 objects 2024-03-24T12:12:12.817 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_5_[83733]: shutting down. 2024-03-24T12:12:12.817 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:12.817 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:12.817 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_6_[83865]: starting. 2024-03-24T12:12:12.818 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_6_[83865]: creating pool ceph_test_rados_list_parallel.smithi006-82538 2024-03-24T12:12:12.818 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_6_[83865]: created object 0... 2024-03-24T12:12:12.818 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_6_[83865]: created object 25... 2024-03-24T12:12:12.818 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_6_[83865]: created object 49... 2024-03-24T12:12:12.818 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_6_[83865]: finishing. 2024-03-24T12:12:12.818 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_6_[83865]: shutting down. 2024-03-24T12:12:12.818 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:12.818 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:12.818 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_7_[83866]: starting. 2024-03-24T12:12:12.818 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_7_[83866]: listing objects. 2024-03-24T12:12:12.818 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_7_[83866]: listed object 0... 2024-03-24T12:12:12.818 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_7_[83866]: listed object 25... 2024-03-24T12:12:12.818 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_7_[83866]: listed object 50... 2024-03-24T12:12:12.818 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_7_[83866]: saw 56 objects 2024-03-24T12:12:12.818 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_7_[83866]: shutting down. 2024-03-24T12:12:12.819 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:12.819 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:12.819 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_8_[83867]: starting. 2024-03-24T12:12:12.819 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_8_[83867]: added 25 objects... 2024-03-24T12:12:12.819 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_8_[83867]: added half of the objects 2024-03-24T12:12:12.819 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_8_[83867]: added 50 objects... 2024-03-24T12:12:12.819 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_8_[83867]: added 50 objects 2024-03-24T12:12:12.819 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_8_[83867]: shutting down. 2024-03-24T12:12:12.819 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:12.819 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:13.146 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ********************** api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2024-03-24T12:12:13.146 INFO:tasks.workunit.client.0.smithi006.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2024-03-24T12:12:13.146 INFO:tasks.workunit.client.0.smithi006.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2024-03-24T12:12:13.146 INFO:tasks.workunit.client.0.smithi006.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2024-03-24T12:12:13.147 INFO:tasks.workunit.client.0.smithi006.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (11493 ms) 2024-03-24T12:12:13.147 INFO:tasks.workunit.client.0.smithi006.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (11493 ms total) 2024-03-24T12:12:13.147 INFO:tasks.workunit.client.0.smithi006.stdout: api_cls_remote_reads: 2024-03-24T12:12:13.147 INFO:tasks.workunit.client.0.smithi006.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2024-03-24T12:12:13.147 INFO:tasks.workunit.client.0.smithi006.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (11493 ms total) 2024-03-24T12:12:13.147 INFO:tasks.workunit.client.0.smithi006.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2024-03-24T12:12:13.157 INFO:tasks.workunit.client.0.smithi006.stdout: handler_error: Running main() from gmock_main.cc 2024-03-24T12:12:13.157 INFO:tasks.workunit.client.0.smithi006.stdout: handler_error: [==========] Running 1 test from 1 test suite. 2024-03-24T12:12:13.157 INFO:tasks.workunit.client.0.smithi006.stdout: handler_error: [----------] Global test environment set-up. 2024-03-24T12:12:13.157 INFO:tasks.workunit.client.0.smithi006.stdout: handler_error: [----------] 1 test from neocls_handler_error 2024-03-24T12:12:13.158 INFO:tasks.workunit.client.0.smithi006.stdout: handler_error: [ RUN ] neocls_handler_error.test_handler_error 2024-03-24T12:12:13.158 INFO:tasks.workunit.client.0.smithi006.stdout: handler_error: [ OK ] neocls_handler_error.test_handler_error (11396 ms) 2024-03-24T12:12:13.158 INFO:tasks.workunit.client.0.smithi006.stdout: handler_error: [----------] 1 test from neocls_handler_error (11396 ms total) 2024-03-24T12:12:13.158 INFO:tasks.workunit.client.0.smithi006.stdout: handler_error: 2024-03-24T12:12:13.158 INFO:tasks.workunit.client.0.smithi006.stdout: handler_error: [----------] Global test environment tear-down 2024-03-24T12:12:13.158 INFO:tasks.workunit.client.0.smithi006.stdout: handler_error: [==========] 1 test from 1 test suite ran. (11397 ms total) 2024-03-24T12:12:13.158 INFO:tasks.workunit.client.0.smithi006.stdout: handler_error: [ PASSED ] 1 test. 2024-03-24T12:12:13.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82516-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi006-81832-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi006-82159-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi006-82137-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi006-82543-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi006-82061-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi006-82180-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:13.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82421-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:12 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1149679251' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi006-82271-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:13.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:12 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:12:13.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:12 smithi090 ceph-mon[33646]: osdmap e78: 8 total, 8 up, 8 in 2024-03-24T12:12:13.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi006-82271-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:13.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:12 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3619165262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:13.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:12 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:13.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82516-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi006-81832-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi006-82159-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi006-82137-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi006-82543-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi006-82061-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi006-82180-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:13.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82421-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1149679251' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi006-82271-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:13.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:12:13.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[41734]: osdmap e78: 8 total, 8 up, 8 in 2024-03-24T12:12:13.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi006-82271-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:13.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3619165262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:13.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:13.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82516-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi006-81832-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi006-82159-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi006-82137-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi006-82543-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi006-82061-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi006-82180-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:13.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82421-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:13.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1149679251' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi006-82271-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:13.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:12:13.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[30108]: osdmap e78: 8 total, 8 up, 8 in 2024-03-24T12:12:13.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi006-82271-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:13.214 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3619165262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:13.214 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:12 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:13.306 INFO:tasks.workunit.client.0.smithi006.stdout:********* 2024-03-24T12:12:13.306 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_9_[83913]: starting. 2024-03-24T12:12:13.306 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_9_[83913]: creating pool ceph_test_rados_list_parallel.smithi006-82538 2024-03-24T12:12:13.306 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_9_[83913]: created object 0... 2024-03-24T12:12:13.306 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_9_[83913]: created object 25... 2024-03-24T12:12:13.306 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_9_[83913]: created object 49... 2024-03-24T12:12:13.306 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_9_[83913]: finishing. 2024-03-24T12:12:13.306 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_9_[83913]: shutting down. 2024-03-24T12:12:13.306 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:13.306 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:13.306 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:13.307 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_10_[83914]: starting. 2024-03-24T12:12:13.307 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_10_[83914]: listing objects. 2024-03-24T12:12:13.307 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_10_[83914]: listed object 0... 2024-03-24T12:12:13.307 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_10_[83914]: listed object 25... 2024-03-24T12:12:13.307 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_10_[83914]: listed object 50... 2024-03-24T12:12:13.307 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_10_[83914]: listed object 75... 2024-03-24T12:12:13.307 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_10_[83914]: saw 97 objects 2024-03-24T12:12:13.307 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_10_[83914]: shutting down. 2024-03-24T12:12:13.307 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:13.307 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:13.307 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:13.307 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_13_[83917]: starting. 2024-03-24T12:12:13.307 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_13_[83917]: removed 25 objects... 2024-03-24T12:12:13.307 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_13_[83917]: removed half of the objects 2024-03-24T12:12:13.308 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_13_[83917]: removed 50 objects... 2024-03-24T12:12:13.308 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_13_[83917]: removed 50 objects 2024-03-24T12:12:13.308 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_13_[83917]: shutting down. 2024-03-24T12:12:13.308 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:13.308 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:13.308 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:13.308 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_11_[83915]: starting. 2024-03-24T12:12:13.308 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_11_[83915]: added 25 objects... 2024-03-24T12:12:13.308 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_11_[83915]: added half of the objects 2024-03-24T12:12:13.308 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_11_[83915]: added 50 objects... 2024-03-24T12:12:13.308 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_11_[83915]: added 50 objects 2024-03-24T12:12:13.308 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_11_[83915]: shutting down. 2024-03-24T12:12:13.308 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:13.308 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:13.309 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:13.309 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_12_[83916]: starting. 2024-03-24T12:12:13.309 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_12_[83916]: added 25 objects... 2024-03-24T12:12:13.309 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_12_[83916]: added half of the objects 2024-03-24T12:12:13.309 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_12_[83916]: added 50 objects... 2024-03-24T12:12:13.309 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_12_[83916]: added 50 objects 2024-03-24T12:12:13.309 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_12_[83916]: shutting down. 2024-03-24T12:12:13.309 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:13.309 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:13.309 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:13.309 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:13.309 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_14_[84002]: starting. 2024-03-24T12:12:13.309 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_14_[84002]: creating pool ceph_test_rados_list_parallel.smithi006-82538 2024-03-24T12:12:13.309 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_14_[84002]: created object 0... 2024-03-24T12:12:13.309 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_14_[84002]: created object 25... 2024-03-24T12:12:13.310 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_14_[84002]: created object 49... 2024-03-24T12:12:13.310 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_14_[84002]: finishing. 2024-03-24T12:12:13.310 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_14_[84002]: shutting down. 2024-03-24T12:12:13.310 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:13.310 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:13.310 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:13.310 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:13.310 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_15_[84003]: starting. 2024-03-24T12:12:13.310 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_15_[84003]: listing objects. 2024-03-24T12:12:13.310 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_15_[84003]: listed object 0... 2024-03-24T12:12:14.016 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_15_[84003]: listed object 2 api_c_read_operations: Running main() from gmock_main.cc 2024-03-24T12:12:14.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2024-03-24T12:12:14.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-03-24T12:12:14.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2024-03-24T12:12:14.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-03-24T12:12:14.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-03-24T12:12:14.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-03-24T12:12:14.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (355 ms) 2024-03-24T12:12:14.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-03-24T12:12:14.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (8 ms) 2024-03-24T12:12:14.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-03-24T12:12:14.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (9 ms) 2024-03-24T12:12:14.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-03-24T12:12:14.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (29 ms) 2024-03-24T12:12:14.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-03-24T12:12:14.018 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (9 ms) 2024-03-24T12:12:14.018 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-03-24T12:12:14.018 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (8 ms) 2024-03-24T12:12:14.018 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-03-24T12:12:14.018 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (7 ms) 2024-03-24T12:12:14.018 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-03-24T12:12:14.018 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (5 ms) 2024-03-24T12:12:14.018 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-03-24T12:12:14.018 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (4 ms) 2024-03-24T12:12:14.018 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-03-24T12:12:14.018 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (5 ms) 2024-03-24T12:12:14.018 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-03-24T12:12:14.018 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (6 ms) 2024-03-24T12:12:14.018 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2024-03-24T12:12:14.018 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (5 ms) 2024-03-24T12:12:14.018 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-03-24T12:12:14.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (12 ms) 2024-03-24T12:12:14.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-03-24T12:12:14.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (9 ms) 2024-03-24T12:12:14.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-03-24T12:12:14.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (8 ms) 2024-03-24T12:12:14.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-03-24T12:12:14.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (4 ms) 2024-03-24T12:12:14.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (483 ms total) 2024-03-24T12:12:14.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: 2024-03-24T12:12:14.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-03-24T12:12:14.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (12396 ms total) 2024-03-24T12:12:14.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2024-03-24T12:12:14.021 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd_pp: Running main() from gmock_main.cc 2024-03-24T12:12:14.021 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-03-24T12:12:14.021 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-03-24T12:12:14.021 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-03-24T12:12:14.021 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-03-24T12:12:14.021 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (963 ms) 2024-03-24T12:12:14.021 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-03-24T12:12:14.021 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (2050 ms) 2024-03-24T12:12:14.021 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-03-24T12:12:14.021 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (9429 ms) 2024-03-24T12:12:14.022 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (12442 ms total) 2024-03-24T12:12:14.022 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd_pp: 2024-03-24T12:12:14.022 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-03-24T12:12:14.022 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (12443 ms total) 2024-03-24T12:12:14.022 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-03-24T12:12:14.032 INFO:tasks.workunit.client.0.smithi006.stdout:5... 2024-03-24T12:12:14.032 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_15_[84003]: listed object 50... 2024-03-24T12:12:14.032 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_15_[84003]: listed object 75... 2024-03-24T12:12:14.033 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_15_[84003]: listed object 100... 2024-03-24T12:12:14.033 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_15_[84003]: listed object 125... 2024-03-24T12:12:14.033 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_15_[84003]: saw 150 objects 2024-03-24T12:12:14.033 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_15_[84003]: shutting down. 2024-03-24T12:12:14.033 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:14.033 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:14.033 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:14.033 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:14.033 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_16_[84004]: starting. 2024-03-24T12:12:14.033 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_16_[84004]: added 25 objects... 2024-03-24T12:12:14.033 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_16_[84004]: added half of the objects 2024-03-24T12:12:14.033 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_16_[84004]: added 50 objects... 2024-03-24T12:12:14.033 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_16_[84004]: added 50 objects 2024-03-24T12:12:14.033 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: process_16_[84004]: shutting down. 2024-03-24T12:12:14.033 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:14.034 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:14.034 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:14.034 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:14.034 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******************************* 2024-03-24T12:12:14.034 INFO:tasks.workunit.client.0.smithi006.stdout: list_parallel: ******* SUCCESS ********** 2024-03-24T12:12:14.308 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: Running main() from gmock_main.cc 2024-03-24T12:12:14.308 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-03-24T12:12:14.308 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: [----------] Global test environment set-up. 2024-03-24T12:12:14.308 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-03-24T12:12:14.308 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-03-24T12:12:14.308 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (83 ms) 2024-03-24T12:12:14.308 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-03-24T12:12:14.308 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (71 ms) 2024-03-24T12:12:14.308 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-03-24T12:12:14.308 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (11449 ms) 2024-03-24T12:12:14.308 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-03-24T12:12:14.308 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:11.814267+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82516-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:14.309 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:11.814328+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:14.309 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:11.814372+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi006-81832-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:14.309 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:11.814413+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi006-82159-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:14.309 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:11.814457+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi006-82137-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:14.309 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:11.814500+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi006-82543-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:14.309 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:11.814544+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi006-82061-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:14.309 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:11.814585+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi006-82180-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:14.309 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:11.814672+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:14.309 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:11.814771+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:14.309 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:11.814860+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82421-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:14.309 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:11.822389+0000 mon.b [INF] from='client.? 172.21.15.6:0/1149679251' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi006-82271-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi006-82783-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi006-82783-1"}]': finished 2024-03-24T12:12:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:14 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi006-82972-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi006-82972-1"}]': finished 2024-03-24T12:12:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:14 smithi090 ceph-mon[33646]: osdmap e79: 8 total, 8 up, 8 in 2024-03-24T12:12:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3863913256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi006-81794-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:14 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi006-81794-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3775491415' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-24T12:12:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4087805071' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:15.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3591427439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:15.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:15.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi006-82783-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi006-82783-1"}]': finished 2024-03-24T12:12:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:14 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi006-82972-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi006-82972-1"}]': finished 2024-03-24T12:12:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:14 smithi006 ceph-mon[30108]: osdmap e79: 8 total, 8 up, 8 in 2024-03-24T12:12:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3863913256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi006-81794-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:14 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi006-81794-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3775491415' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-24T12:12:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4087805071' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3591427439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi006-82783-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi006-82783-1"}]': finished 2024-03-24T12:12:15.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:14 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi006-82972-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi006-82972-1"}]': finished 2024-03-24T12:12:15.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:14 smithi006 ceph-mon[41734]: osdmap e79: 8 total, 8 up, 8 in 2024-03-24T12:12:15.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3863913256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi006-81794-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:15.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:14 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi006-81794-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:15.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3775491415' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-24T12:12:15.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4087805071' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:15.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3591427439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:15.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:15.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:12:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:12:15.874 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:12.461988+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi006-82271-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:15.874 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:12.500905+0000 mon.b [INF] from='client.? 172.21.15.6:0/3619165262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:15.874 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:12.826652+0000 mon.a [INF] from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi006-82783-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi006-82783-1"}]': finished 2024-03-24T12:12:15.874 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:12.826742+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi006-82972-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi006-82972-1"}]': finished 2024-03-24T12:12:15.875 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:13.143719+0000 mon.c [INF] from='client.? 172.21.15.6:0/3863913256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi006-81794-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:15.875 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:13.154539+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi006-81794-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:15.875 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:13.205672+0000 mon.b [INF] from='client.? 172.21.15.6:0/4087805071' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:15.875 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:13.209017+0000 mon.b [INF] from='client.? 172.21.15.6:0/3591427439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:15.875 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:13.448508+0000 client.admin [INF] onexx 2024-03-24T12:12:15.875 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:14.005698+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:15.875 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:14.006104+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi006-82271-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]': finished 2024-03-24T12:12:15.875 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:14.006223+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi006-81794-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:15.875 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:14.012916+0000 mon.b [INF] from='client.? 172.21.15.6:0/2845973739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:15.875 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:14.309197+0000 mon.b [INF] from='client.? 172.21.15.6:0/3619165262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:15.875 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:14.695160+0000 mon.b [INF] from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.0"}]: dispatch 2024-03-24T12:12:15.875 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:14.695825+0000 mon.b [INF] from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.1"}]: dispatch 2024-03-24T12:12:16.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:15 smithi090 ceph-mon[33646]: pgmap v76: 788 pgs: 6 active, 189 creating+activating, 62 creating+peering, 437 active+clean, 94 unknown; 583 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-24T12:12:16.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:15 smithi090 ceph-mon[33646]: onexx 2024-03-24T12:12:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:15 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:15 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi006-82271-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]': finished 2024-03-24T12:12:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:15 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi006-81794-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2845973739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:15 smithi090 ceph-mon[33646]: osdmap e80: 8 total, 8 up, 8 in 2024-03-24T12:12:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3619165262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.0"}]: dispatch 2024-03-24T12:12:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.1"}]: dispatch 2024-03-24T12:12:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.2"}]: dispatch 2024-03-24T12:12:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.3"}]: dispatch 2024-03-24T12:12:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.4"}]: dispatch 2024-03-24T12:12:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.5"}]: dispatch 2024-03-24T12:12:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.6"}]: dispatch 2024-03-24T12:12:16.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.7"}]: dispatch 2024-03-24T12:12:16.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.8"}]: dispatch 2024-03-24T12:12:16.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.9"}]: dispatch 2024-03-24T12:12:16.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:15 smithi090 ceph-mon[33646]: Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:16.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-24T12:12:16.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi006-82783-1"}]: dispatch 2024-03-24T12:12:16.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[41734]: pgmap v76: 788 pgs: 6 active, 189 creating+activating, 62 creating+peering, 437 active+clean, 94 unknown; 583 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-24T12:12:16.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[41734]: onexx 2024-03-24T12:12:16.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:16.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi006-82271-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]': finished 2024-03-24T12:12:16.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi006-81794-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:16.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2845973739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:16.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[41734]: osdmap e80: 8 total, 8 up, 8 in 2024-03-24T12:12:16.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3619165262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:16.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:16.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.0"}]: dispatch 2024-03-24T12:12:16.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.1"}]: dispatch 2024-03-24T12:12:16.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.2"}]: dispatch 2024-03-24T12:12:16.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.3"}]: dispatch 2024-03-24T12:12:16.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.4"}]: dispatch 2024-03-24T12:12:16.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.5"}]: dispatch 2024-03-24T12:12:16.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.6"}]: dispatch 2024-03-24T12:12:16.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.7"}]: dispatch 2024-03-24T12:12:16.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.8"}]: dispatch 2024-03-24T12:12:16.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.9"}]: dispatch 2024-03-24T12:12:16.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[41734]: Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:16.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-24T12:12:16.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi006-82783-1"}]: dispatch 2024-03-24T12:12:16.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[30108]: pgmap v76: 788 pgs: 6 active, 189 creating+activating, 62 creating+peering, 437 active+clean, 94 unknown; 583 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-24T12:12:16.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[30108]: onexx 2024-03-24T12:12:16.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:16.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi006-82271-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]': finished 2024-03-24T12:12:16.216 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi006-81794-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:16.216 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2845973739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:16.216 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[30108]: osdmap e80: 8 total, 8 up, 8 in 2024-03-24T12:12:16.217 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3619165262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:16.217 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:16.217 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.0"}]: dispatch 2024-03-24T12:12:16.217 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.1"}]: dispatch 2024-03-24T12:12:16.217 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.2"}]: dispatch 2024-03-24T12:12:16.218 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.3"}]: dispatch 2024-03-24T12:12:16.218 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.4"}]: dispatch 2024-03-24T12:12:16.218 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.5"}]: dispatch 2024-03-24T12:12:16.218 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.6"}]: dispatch 2024-03-24T12:12:16.218 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.7"}]: dispatch 2024-03-24T12:12:16.219 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.8"}]: dispatch 2024-03-24T12:12:16.219 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.9"}]: dispatch 2024-03-24T12:12:16.219 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[30108]: Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:16.219 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-24T12:12:16.219 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi006-82783-1"}]: dispatch 2024-03-24T12:12:17.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.0"}]: dispatch 2024-03-24T12:12:17.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.1"}]: dispatch 2024-03-24T12:12:17.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.2"}]: dispatch 2024-03-24T12:12:17.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.3"}]: dispatch 2024-03-24T12:12:17.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.4"}]: dispatch 2024-03-24T12:12:17.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.5"}]: dispatch 2024-03-24T12:12:17.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.6"}]: dispatch 2024-03-24T12:12:17.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.7"}]: dispatch 2024-03-24T12:12:17.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.8"}]: dispatch 2024-03-24T12:12:17.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.9"}]: dispatch 2024-03-24T12:12:17.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: 21.4 deep-scrub starts 2024-03-24T12:12:17.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: 21.4 deep-scrub ok 2024-03-24T12:12:17.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: 21.2 deep-scrub starts 2024-03-24T12:12:17.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: 21.2 deep-scrub ok 2024-03-24T12:12:17.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: pgmap v78: 628 pgs: 6 active, 102 creating+activating, 43 creating+peering, 300 active+clean, 177 unknown; 583 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 969 B/s wr, 2 op/s 2024-03-24T12:12:17.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2356903427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:17.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3863913256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi006-81794-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:17.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: 21.5 deep-scrub starts 2024-03-24T12:12:17.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: 21.5 deep-scrub ok 2024-03-24T12:12:17.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: 21.9 deep-scrub starts 2024-03-24T12:12:17.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: 21.9 deep-scrub ok 2024-03-24T12:12:17.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:17.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: osdmap e81: 8 total, 8 up, 8 in 2024-03-24T12:12:17.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-03-24T12:12:17.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:17.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi006-81794-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:17.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi006-82783-1"}]': finished 2024-03-24T12:12:17.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: osdmap e82: 8 total, 8 up, 8 in 2024-03-24T12:12:17.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2356903427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:17.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi006-82159-7"}]: dispatch 2024-03-24T12:12:17.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:17.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3205672555' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi006-81775-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:17.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2356903427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi006-81943-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:17.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi006-82783-1"}]: dispatch 2024-03-24T12:12:17.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi006-82159-7"}]: dispatch 2024-03-24T12:12:17.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3074754154' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:17.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi006-82159-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:17.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi006-81943-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:17.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:17.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:17.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:17.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.0"}]: dispatch 2024-03-24T12:12:17.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.1"}]: dispatch 2024-03-24T12:12:17.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.2"}]: dispatch 2024-03-24T12:12:17.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.3"}]: dispatch 2024-03-24T12:12:17.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.4"}]: dispatch 2024-03-24T12:12:17.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.5"}]: dispatch 2024-03-24T12:12:17.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.6"}]: dispatch 2024-03-24T12:12:17.466 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.7"}]: dispatch 2024-03-24T12:12:17.466 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.8"}]: dispatch 2024-03-24T12:12:17.466 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.9"}]: dispatch 2024-03-24T12:12:17.466 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: 21.4 deep-scrub starts 2024-03-24T12:12:17.466 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: 21.4 deep-scrub ok 2024-03-24T12:12:17.466 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: 21.2 deep-scrub starts 2024-03-24T12:12:17.466 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: 21.2 deep-scrub ok 2024-03-24T12:12:17.468 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: pgmap v78: 628 pgs: 6 active, 102 creating+activating, 43 creating+peering, 300 active+clean, 177 unknown; 583 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 969 B/s wr, 2 op/s 2024-03-24T12:12:17.468 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2356903427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:17.468 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3863913256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi006-81794-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:17.468 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: 21.5 deep-scrub starts 2024-03-24T12:12:17.468 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: 21.5 deep-scrub ok 2024-03-24T12:12:17.468 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: 21.9 deep-scrub starts 2024-03-24T12:12:17.468 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: 21.9 deep-scrub ok 2024-03-24T12:12:17.468 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:17.468 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: osdmap e81: 8 total, 8 up, 8 in 2024-03-24T12:12:17.468 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-03-24T12:12:17.468 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:17.468 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi006-81794-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:17.468 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi006-82783-1"}]': finished 2024-03-24T12:12:17.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: osdmap e82: 8 total, 8 up, 8 in 2024-03-24T12:12:17.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2356903427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:17.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi006-82159-7"}]: dispatch 2024-03-24T12:12:17.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:17.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3205672555' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi006-81775-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:17.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2356903427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi006-81943-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:17.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi006-82783-1"}]: dispatch 2024-03-24T12:12:17.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi006-82159-7"}]: dispatch 2024-03-24T12:12:17.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3074754154' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:17.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi006-82159-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:17.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi006-81943-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:17.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:17.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:17.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:17.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.0"}]: dispatch 2024-03-24T12:12:17.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.1"}]: dispatch 2024-03-24T12:12:17.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.2"}]: dispatch 2024-03-24T12:12:17.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.3"}]: dispatch 2024-03-24T12:12:17.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.4"}]: dispatch 2024-03-24T12:12:17.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.5"}]: dispatch 2024-03-24T12:12:17.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.6"}]: dispatch 2024-03-24T12:12:17.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.7"}]: dispatch 2024-03-24T12:12:17.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.8"}]: dispatch 2024-03-24T12:12:17.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.9"}]: dispatch 2024-03-24T12:12:17.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: 21.4 deep-scrub starts 2024-03-24T12:12:17.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: 21.4 deep-scrub ok 2024-03-24T12:12:17.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: 21.2 deep-scrub starts 2024-03-24T12:12:17.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: 21.2 deep-scrub ok 2024-03-24T12:12:17.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: pgmap v78: 628 pgs: 6 active, 102 creating+activating, 43 creating+peering, 300 active+clean, 177 unknown; 583 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 969 B/s wr, 2 op/s 2024-03-24T12:12:17.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2356903427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:17.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3863913256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi006-81794-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:17.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: 21.5 deep-scrub starts 2024-03-24T12:12:17.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: 21.5 deep-scrub ok 2024-03-24T12:12:17.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: 21.9 deep-scrub starts 2024-03-24T12:12:17.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: 21.9 deep-scrub ok 2024-03-24T12:12:17.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:17.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: osdmap e81: 8 total, 8 up, 8 in 2024-03-24T12:12:17.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-03-24T12:12:17.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:17.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi006-81794-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:17.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi006-82783-1"}]': finished 2024-03-24T12:12:17.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: osdmap e82: 8 total, 8 up, 8 in 2024-03-24T12:12:17.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2356903427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:17.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi006-82159-7"}]: dispatch 2024-03-24T12:12:17.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:17.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3205672555' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi006-81775-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:17.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2356903427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi006-81943-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:17.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi006-82783-1"}]: dispatch 2024-03-24T12:12:17.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi006-82159-7"}]: dispatch 2024-03-24T12:12:17.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3074754154' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:17.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi006-82159-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:17.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi006-81943-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:17.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:17.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:17.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:18.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: 21.6 deep-scrub starts 2024-03-24T12:12:18.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: 21.6 deep-scrub ok 2024-03-24T12:12:18.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: 21.0 deep-scrub starts 2024-03-24T12:12:18.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: 21.0 deep-scrub ok 2024-03-24T12:12:18.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: 21.8 deep-scrub starts 2024-03-24T12:12:18.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: 21.8 deep-scrub ok 2024-03-24T12:12:18.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: 21.7 deep-scrub starts 2024-03-24T12:12:18.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: 21.7 deep-scrub ok 2024-03-24T12:12:18.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: 21.3 deep-scrub starts 2024-03-24T12:12:18.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: 21.3 deep-scrub ok 2024-03-24T12:12:18.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4087805071' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:18.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3591427439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:18.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: 21.1 deep-scrub starts 2024-03-24T12:12:18.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: 21.1 deep-scrub ok 2024-03-24T12:12:18.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: twoxx 2024-03-24T12:12:18.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:18.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:12:18.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: pgmap v81: 884 pgs: 6 active, 63 creating+activating, 118 creating+peering, 366 active+clean, 331 unknown; 49 MiB data, 401 MiB used, 715 GiB / 715 GiB avail; 5.7 MiB/s rd, 11 MiB/s wr, 490 op/s 2024-03-24T12:12:18.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-03-24T12:12:18.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:18.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:18.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi006-81794-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:18.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi006-82783-1"}]': finished 2024-03-24T12:12:18.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3074754154' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:18.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi006-82159-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:18.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi006-81943-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:18.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2356903427' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi006-81943-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:18.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3863913256' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi006-81794-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-24T12:12:18.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: 21.6 deep-scrub starts 2024-03-24T12:12:18.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: 21.6 deep-scrub ok 2024-03-24T12:12:18.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: 21.0 deep-scrub starts 2024-03-24T12:12:18.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: 21.0 deep-scrub ok 2024-03-24T12:12:18.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: 21.8 deep-scrub starts 2024-03-24T12:12:18.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: 21.8 deep-scrub ok 2024-03-24T12:12:18.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: 21.7 deep-scrub starts 2024-03-24T12:12:18.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: 21.7 deep-scrub ok 2024-03-24T12:12:18.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: 21.3 deep-scrub starts 2024-03-24T12:12:18.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: 21.3 deep-scrub ok 2024-03-24T12:12:18.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4087805071' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:18.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3591427439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:18.466 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: 21.1 deep-scrub starts 2024-03-24T12:12:18.466 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: 21.1 deep-scrub ok 2024-03-24T12:12:18.466 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: twoxx 2024-03-24T12:12:18.466 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:18.466 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:12:18.467 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: pgmap v81: 884 pgs: 6 active, 63 creating+activating, 118 creating+peering, 366 active+clean, 331 unknown; 49 MiB data, 401 MiB used, 715 GiB / 715 GiB avail; 5.7 MiB/s rd, 11 MiB/s wr, 490 op/s 2024-03-24T12:12:18.467 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-03-24T12:12:18.467 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:18.467 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:18.467 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi006-81794-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:18.467 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi006-82783-1"}]': finished 2024-03-24T12:12:18.467 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3074754154' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:18.467 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi006-82159-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:18.468 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi006-81943-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:18.468 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2356903427' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi006-81943-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:18.468 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3863913256' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi006-81794-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-24T12:12:18.468 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2548144870' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:18.468 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3974774221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi006-82783-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.468 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: osdmap e83: 8 total, 8 up, 8 in 2024-03-24T12:12:18.468 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3619165262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi006-81813-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.468 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi006-81943-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:18.468 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi006-82159-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi006-82159-7"}]: dispatch 2024-03-24T12:12:18.468 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi006-81794-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-24T12:12:18.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi006-82783-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi006-81775-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:18.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:18.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:18.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4087805071' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:18.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3591427439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi006-82140-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi006-81813-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi006-82140-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.469 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:18.469 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2548144870' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:18.470 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3974774221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi006-82783-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.470 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: osdmap e83: 8 total, 8 up, 8 in 2024-03-24T12:12:18.470 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3619165262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi006-81813-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.470 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi006-81943-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:18.470 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi006-82159-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi006-82159-7"}]: dispatch 2024-03-24T12:12:18.470 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi006-81794-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-24T12:12:18.470 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi006-82783-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.470 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi006-81775-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:18.470 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:18.470 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:18.470 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4087805071' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.470 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:18.471 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3591427439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi006-82140-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.471 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi006-81813-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.471 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.471 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi006-82140-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.471 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:18.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: 21.6 deep-scrub starts 2024-03-24T12:12:18.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: 21.6 deep-scrub ok 2024-03-24T12:12:18.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: 21.0 deep-scrub starts 2024-03-24T12:12:18.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: 21.0 deep-scrub ok 2024-03-24T12:12:18.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: 21.8 deep-scrub starts 2024-03-24T12:12:18.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: 21.8 deep-scrub ok 2024-03-24T12:12:18.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: 21.7 deep-scrub starts 2024-03-24T12:12:18.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: 21.7 deep-scrub ok 2024-03-24T12:12:18.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: 21.3 deep-scrub starts 2024-03-24T12:12:18.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: 21.3 deep-scrub ok 2024-03-24T12:12:18.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4087805071' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:18.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3591427439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:18.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: 21.1 deep-scrub starts 2024-03-24T12:12:18.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: 21.1 deep-scrub ok 2024-03-24T12:12:18.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: twoxx 2024-03-24T12:12:18.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:18.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:12:18.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: pgmap v81: 884 pgs: 6 active, 63 creating+activating, 118 creating+peering, 366 active+clean, 331 unknown; 49 MiB data, 401 MiB used, 715 GiB / 715 GiB avail; 5.7 MiB/s rd, 11 MiB/s wr, 490 op/s 2024-03-24T12:12:18.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-03-24T12:12:18.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:18.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:18.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi006-81794-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:18.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi006-82783-1"}]': finished 2024-03-24T12:12:18.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3074754154' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:18.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi006-82159-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:18.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi006-81943-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:18.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2356903427' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi006-81943-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:18.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3863913256' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi006-81794-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-24T12:12:18.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2548144870' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:18.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3974774221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi006-82783-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: osdmap e83: 8 total, 8 up, 8 in 2024-03-24T12:12:18.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3619165262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi006-81813-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi006-81943-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:18.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi006-82159-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi006-82159-7"}]: dispatch 2024-03-24T12:12:18.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi006-81794-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-24T12:12:18.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi006-82783-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi006-81775-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:18.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:18.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:18.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4087805071' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:18.682 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3591427439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi006-82140-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.682 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi006-81813-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.682 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.682 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi006-82140-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:18.682 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-24T12:12:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi006-81794-3", "field": "max_bytes", "val": "4096"}]': finished 2024-03-24T12:12:19.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi006-82783-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:19.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi006-81775-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:19.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:19.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi006-81813-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:19.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:19.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi006-82140-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:19.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3974774221' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi006-82783-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi006-82783-2"}]: dispatch 2024-03-24T12:12:19.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3619165262' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi006-81813-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:19.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4087805071' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi006-81832-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:19.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3591427439' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi006-82140-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:19.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:19.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:19 smithi090 ceph-mon[33646]: osdmap e84: 8 total, 8 up, 8 in 2024-03-24T12:12:19.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3070372572' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi006-82972-1"}]: dispatch 2024-03-24T12:12:19.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi006-82783-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi006-82783-2"}]: dispatch 2024-03-24T12:12:19.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi006-81813-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:19.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi006-81832-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:19.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi006-82140-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:19.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi006-82972-1"}]: dispatch 2024-03-24T12:12:19.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1967727185' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-24T12:12:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi006-81794-3", "field": "max_bytes", "val": "4096"}]': finished 2024-03-24T12:12:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi006-82783-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi006-81775-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi006-81813-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:19.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi006-82140-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:19.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3974774221' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi006-82783-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi006-82783-2"}]: dispatch 2024-03-24T12:12:19.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3619165262' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi006-81813-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:19.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4087805071' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi006-81832-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:19.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3591427439' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi006-82140-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:19.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:19.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[30108]: osdmap e84: 8 total, 8 up, 8 in 2024-03-24T12:12:19.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3070372572' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi006-82972-1"}]: dispatch 2024-03-24T12:12:19.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi006-82783-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi006-82783-2"}]: dispatch 2024-03-24T12:12:19.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi006-81813-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:19.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi006-81832-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:19.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi006-82140-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:19.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi006-82972-1"}]: dispatch 2024-03-24T12:12:19.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1967727185' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:19.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-24T12:12:19.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:19.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi006-81794-3", "field": "max_bytes", "val": "4096"}]': finished 2024-03-24T12:12:19.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi006-82783-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:19.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi006-81775-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:19.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:19.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi006-81813-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:19.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:19.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi006-82140-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:19.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3974774221' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi006-82783-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi006-82783-2"}]: dispatch 2024-03-24T12:12:19.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3619165262' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi006-81813-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:19.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4087805071' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi006-81832-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:19.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3591427439' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi006-82140-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:19.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:19.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[41734]: osdmap e84: 8 total, 8 up, 8 in 2024-03-24T12:12:19.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3070372572' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi006-82972-1"}]: dispatch 2024-03-24T12:12:19.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi006-82783-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi006-82783-2"}]: dispatch 2024-03-24T12:12:19.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi006-81813-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:19.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi006-81832-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:19.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi006-82140-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:19.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi006-82972-1"}]: dispatch 2024-03-24T12:12:19.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1967727185' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:19.822 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:1 cls: Running main() from gmock_main.cc 2024-03-24T12:12:19.823 INFO:tasks.workunit.client.0.smithi006.stdout: cls: [==========] Running 2 tests from 1 test suite. 2024-03-24T12:12:19.823 INFO:tasks.workunit.client.0.smithi006.stdout: cls: [----------] Global test environment set-up. 2024-03-24T12:12:19.823 INFO:tasks.workunit.client.0.smithi006.stdout: cls: [----------] 2 tests from NeoRadosCls 2024-03-24T12:12:19.823 INFO:tasks.workunit.client.0.smithi006.stdout: cls: [ RUN ] NeoRadosCls.DNE 2024-03-24T12:12:19.823 INFO:tasks.workunit.client.0.smithi006.stdout: cls: [ OK ] NeoRadosCls.DNE (12330 ms) 2024-03-24T12:12:19.823 INFO:tasks.workunit.client.0.smithi006.stdout: cls: [ RUN ] NeoRadosCls.RemoteReads 2024-03-24T12:12:19.823 INFO:tasks.workunit.client.0.smithi006.stdout: cls: [ OK ] NeoRadosCls.RemoteReads (5807 ms) 2024-03-24T12:12:19.823 INFO:tasks.workunit.client.0.smithi006.stdout: cls: [----------] 2 tests from NeoRadosCls (18137 ms total) 2024-03-24T12:12:19.823 INFO:tasks.workunit.client.0.smithi006.stdout: cls: 2024-03-24T12:12:19.823 INFO:tasks.workunit.client.0.smithi006.stdout: cls: [----------] Global test environment tear-down 2024-03-24T12:12:19.823 INFO:tasks.workunit.client.0.smithi006.stdout: cls: [==========] 2 tests from 1 test suite ran. (18137 ms total) 2024-03-24T12:12:19.823 INFO:tasks.workunit.client.0.smithi006.stdout: cls: [ PASSED ] 2 tests. 2024-03-24T12:12:20.083 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: process_1_[82627]: starting. 2024-03-24T12:12:20.084 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: process_1_[82627]: creating pool ceph_test_rados_open_pools_parallel.smithi006-82567 2024-03-24T12:12:20.084 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: process_1_[82627]: created object 0... 2024-03-24T12:12:20.084 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: process_1_[82627]: created object 25... 2024-03-24T12:12:20.084 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: process_1_[82627]: created object 49... 2024-03-24T12:12:20.084 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: process_1_[82627]: finishing. 2024-03-24T12:12:20.084 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: process_1_[82627]: shutting down. 2024-03-24T12:12:20.084 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: process_2_[82628]: starting. 2024-03-24T12:12:20.084 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: process_2_[82628]: rados_pool_create. 2024-03-24T12:12:20.084 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: process_2_[82628]: rados_ioctx_create. 2024-03-24T12:12:20.084 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: process_2_[82628]: shutting down. 2024-03-24T12:12:20.084 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: ******************************* 2024-03-24T12:12:20.084 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: process_3_[84271]: starting. 2024-03-24T12:12:20.084 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: process_3_[84271]: creating pool ceph_test_rados_open_pools_parallel.smithi006-82567 2024-03-24T12:12:20.084 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: process_3_[84271]: created object 0... 2024-03-24T12:12:20.084 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: process_3_[84271]: created object 25... 2024-03-24T12:12:20.085 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: process_3_[84271]: created object 49... 2024-03-24T12:12:20.085 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: process_3_[84271]: finishing. 2024-03-24T12:12:20.085 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: process_3_[84271]: shutting down. 2024-03-24T12:12:20.085 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: ******************************* 2024-03-24T12:12:20.085 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: process_4_[84272]: starting. 2024-03-24T12:12:20.085 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: process_4_[84272]: rados_pool_create. 2024-03-24T12:12:20.085 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: process_4_[84272]: rados_ioctx_create. 2024-03-24T12:12:20.085 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: process_4_[84272]: shutting down. 2024-03-24T12:12:20.085 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: ******************************* 2024-03-24T12:12:20.085 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: ******************************* 2024-03-24T12:12:20.085 INFO:tasks.workunit.client.0.smithi006.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-03-24T12:12:20.341 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_1_[82733]: starting. 2024-03-24T12:12:20.341 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_1_[82733]: creating pool ceph_test_rados_delete_pools_parallel.smithi006-82675 2024-03-24T12:12:20.341 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_1_[82733]: created object 0... 2024-03-24T12:12:20.341 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_1_[82733]: created object 25... 2024-03-24T12:12:20.341 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_1_[82733]: created object 49... 2024-03-24T12:12:20.341 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_1_[82733]: finishing. 2024-03-24T12:12:20.341 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_1_[82733]: shutting down. 2024-03-24T12:12:20.341 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_2_[82751]: starting. 2024-03-24T12:12:20.341 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_2_[82751]: deleting pool ceph_test_rados_delete_pools_parallel.smithi006-82675 2024-03-24T12:12:20.341 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_2_[82751]: shutting down. 2024-03-24T12:12:20.341 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: ******************************* 2024-03-24T12:12:20.341 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_3_[84052]: starting. 2024-03-24T12:12:20.342 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_3_[84052]: creating pool ceph_test_rados_delete_pools_parallel.smithi006-82675 2024-03-24T12:12:20.342 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_3_[84052]: created object 0... 2024-03-24T12:12:20.342 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_3_[84052]: created object 25... 2024-03-24T12:12:20.342 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_3_[84052]: created object 49... 2024-03-24T12:12:20.342 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_3_[84052]: finishing. 2024-03-24T12:12:20.342 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_3_[84052]: shutting down. 2024-03-24T12:12:20.342 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: ******************************* 2024-03-24T12:12:20.342 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_5_[84054]: starting. 2024-03-24T12:12:20.342 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_5_[84054]: listing objects. 2024-03-24T12:12:20.342 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_5_[84054]: listed object 0... 2024-03-24T12:12:20.342 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_5_[84054]: listed object 25... 2024-03-24T12:12:20.342 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_5_[84054]: saw 50 objects 2024-03-24T12:12:20.342 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_5_[84054]: shutting down. 2024-03-24T12:12:20.342 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: ******************************* 2024-03-24T12:12:20.342 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_4_[84053]: starting. 2024-03-24T12:12:20.343 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_4_[84053]: deleting pool ceph_test_rados_delete_pools_parallel.smithi006-82675 2024-03-24T12:12:20.343 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: process_4_[84053]: shutting down. 2024-03-24T12:12:20.343 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: ******************************* 2024-03-24T12:12:20.343 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: ******************************* 2024-03-24T12:12:20.343 INFO:tasks.workunit.client.0.smithi006.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-03-24T12:12:20.602 INFO:tasks.workunit.client.0.smithi006.stdout:4.696301+0000 mon.b [INF] from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.2"}]: dispatch 2024-03-24T12:12:20.602 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:14.696727+0000 mon.b [INF] from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.3"}]: dispatch 2024-03-24T12:12:20.602 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:14.697375+0000 mon.b [INF] from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.4"}]: dispatch 2024-03-24T12:12:20.602 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:14.697856+0000 mon.b [INF] from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.5"}]: dispatch 2024-03-24T12:12:20.602 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:14.698282+0000 mon.b [INF] from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.6"}]: dispatch 2024-03-24T12:12:20.602 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:14.698684+0000 mon.b [INF] from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.7"}]: dispatch 2024-03-24T12:12:20.603 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:14.699081+0000 mon.b [INF] from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.8"}]: dispatch 2024-03-24T12:12:20.603 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:14.699492+0000 mon.b [INF] from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.9"}]: dispatch 2024-03-24T12:12:20.603 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:14.839085+0000 mon.a [WRN] Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:20.603 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:15.095201+0000 mon.a [INF] from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-24T12:12:20.603 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:15.095995+0000 mon.a [INF] from='client.? 172.21.15.6:0/3050319323' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi006-82783-1"}]: dispatch 2024-03-24T12:12:20.603 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:18.183098+0000 mon.a [INF] from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-24T12:12:20.603 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:18.193653+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:20.603 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:18.193734+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi006-81794-3", "field": "max_bytes", "val": "4096"}]': finished 2024-03-24T12:12:20.603 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:18.193837+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi006-82783-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:20.603 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:18.193961+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi006-81775-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:20.603 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:18.194050+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:20.603 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:18.194131+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi006-81813-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:20.605 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:18.194183+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:20.605 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:18.194230+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi006-82140-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:20.605 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:18.199916+0000 mon.c [INF] from='client.? 172.21.15.6:0/3974774221' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi006-82783-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi006-82783-2"}]: dispatch 2024-03-24T12:12:20.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:18.201450+0000 mon.b [INF] from='client.? 172.21.15.6:0/3619165262' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi006-81813-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:20.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:18.201833+0000 mon.b [INF] from='client.? 172.21.15.6:0/4087805071' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi006-81832-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:20.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:18.202702+0000 mon.b [INF] from='client.? 172.21.15.6:0/3591427439' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi006-82140-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:20.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:18.756655+0000 mon.c [INF] from='client.? 172.21.15.6:0/3070372572' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi006-82972-1"}]: dispatch 2024-03-24T12:12:20.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:18.758378+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi006-82783-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi006-82783-2"}]: dispatch 2024-03-24T12:12:20.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:18.759265+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi006-81813-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:20.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:18.764597+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi006-81832-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:20.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:18.766206+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi006-82140-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:20.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:18.767203+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi006-82972-1"}]: dispatch 2024-03-24T12:12:20.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:18.769539+0000 mon.c [INF] from='client.? 172.21.15.6:0/1967727185' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:20.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:19.283003+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:20.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:20 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:20.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:20 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi006-81943-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi006-81943-10"}]': finished 2024-03-24T12:12:20.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:20 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:20.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:20 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi006-81943-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi006-81943-10"}]': finished 2024-03-24T12:12:20.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:20 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:20.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:20 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi006-81943-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi006-81943-10"}]': finished 2024-03-24T12:12:21.610 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:19.288852+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi006-81943-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi006-81943-10"}]': finished 2024-03-24T12:12:21.610 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:19.288939+0000 mon.a [INF] from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi006-82159-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi006-82159-7"}]': finished 2024-03-24T12:12:21.610 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:19.288989+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi006-82972-1"}]': finished 2024-03-24T12:12:21.610 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:19.294455+0000 mon.c [INF] from='client.? 172.21.15.6:0/3070372572' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi006-82972-1"}]: dispatch 2024-03-24T12:12:21.610 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:20.183702+0000 client.admin [INF] threexx 2024-03-24T12:12:21.610 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:20.338581+0000 mon.c [INF] from='client.? 172.21.15.6:0/1967727185' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:21.610 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:20.338987+0000 mon.b [INF] from='client.? 172.21.15.6:0/2845973739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:21.611 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:20.597457+0000 mon.a [WRN] Health check update: 17 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:21.611 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:20.603461+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi006-82972-1"}]: dispatch 2024-03-24T12:12:21.611 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:20.604196+0000 mon.a [INF] from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-24T12:12:21.611 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:20.605451+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:21.611 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:20.612248+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi006-82783-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi006-82783-2"}]': finished 2024-03-24T12:12:21.611 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:20.612313+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi006-81813-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi006-81813-16"}]': finished 2024-03-24T12:12:21.611 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:20.612363+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi006-81832-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi006-81832-22"}]': finished 2024-03-24T12:12:21.611 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:20.612441+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi006-82140-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi006-82140-7"}]': finished 2024-03-24T12:12:21.611 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:20.621574+0000 mon.c [INF] from='client.? 172.21.15.6:0/2547004740' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:21.825 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:20.621921+0000 mon api_io_pp: Running main() from gmock_main.cc 2024-03-24T12:12:21.825 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-03-24T12:12:21.825 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [----------] Global test environment set-up. 2024-03-24T12:12:21.825 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-03-24T12:12:21.826 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: seed 81832 2024-03-24T12:12:21.826 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-03-24T12:12:21.826 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-03-24T12:12:21.826 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-03-24T12:12:21.826 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (354 ms) 2024-03-24T12:12:21.826 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-03-24T12:12:21.826 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (15 ms) 2024-03-24T12:12:21.826 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-03-24T12:12:21.826 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (5 ms) 2024-03-24T12:12:21.827 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-03-24T12:12:21.827 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (2 ms) 2024-03-24T12:12:21.827 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-03-24T12:12:21.827 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (3 ms) 2024-03-24T12:12:21.827 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-03-24T12:12:21.827 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (4 ms) 2024-03-24T12:12:21.827 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-03-24T12:12:21.827 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (3 ms) 2024-03-24T12:12:21.828 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-03-24T12:12:21.828 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (9 ms) 2024-03-24T12:12:21.828 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-03-24T12:12:21.828 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (5 ms) 2024-03-24T12:12:21.828 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-03-24T12:12:21.828 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (3 ms) 2024-03-24T12:12:21.828 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-03-24T12:12:21.828 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (5 ms) 2024-03-24T12:12:21.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-03-24T12:12:21.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (5 ms) 2024-03-24T12:12:21.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-03-24T12:12:21.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (4 ms) 2024-03-24T12:12:21.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-03-24T12:12:21.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (4 ms) 2024-03-24T12:12:21.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-03-24T12:12:21.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (14 ms) 2024-03-24T12:12:21.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-03-24T12:12:21.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (5 ms) 2024-03-24T12:12:21.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-03-24T12:12:21.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (6 ms) 2024-03-24T12:12:21.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-03-24T12:12:21.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (2 ms) 2024-03-24T12:12:21.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-03-24T12:12:21.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (6 ms) 2024-03-24T12:12:21.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (454 ms total) 2024-03-24T12:12:21.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: 2024-03-24T12:12:21.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-03-24T12:12:21.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-03-24T12:12:21.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (1148 ms) 2024-03-24T12:12:21.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-03-24T12:12:21.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (44 ms) 2024-03-24T12:12:21.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-03-24T12:12:21.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (5 ms) 2024-03-24T12:12:21.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-03-24T12:12:21.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (5 ms) 2024-03-24T12:12:21.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-03-24T12:12:21.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (3 ms) 2024-03-24T12:12:21.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: pgmap v84: 876 pgs: 160 unknown, 241 creating+peering, 475 active+clean; 145 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 546 op/s 2024-03-24T12:12:21.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi006-82159-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi006-82159-7"}]': finished 2024-03-24T12:12:21.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi006-82972-1"}]': finished 2024-03-24T12:12:21.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3070372572' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi006-82972-1"}]: dispatch 2024-03-24T12:12:21.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:21.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: threexx 2024-03-24T12:12:21.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1967727185' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:21.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2845973739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:21.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: osdmap e85: 8 total, 8 up, 8 in 2024-03-24T12:12:21.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: Health check update: 17 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:21.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi006-82972-1"}]: dispatch 2024-03-24T12:12:21.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-24T12:12:21.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:21.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi006-82783-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi006-82783-2"}]': finished 2024-03-24T12:12:21.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi006-81813-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi006-81813-16"}]': finished 2024-03-24T12:12:21.931 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi006-81832-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi006-81832-22"}]': finished 2024-03-24T12:12:21.931 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi006-82140-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi006-82140-7"}]': finished 2024-03-24T12:12:21.931 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2547004740' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:21.931 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1967727185' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:21.931 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:21.931 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: pgmap v87: 892 pgs: 304 unknown, 145 creating+peering, 443 active+clean; 145 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 6.0 MiB/s rd, 25 MiB/s wr, 28 op/s 2024-03-24T12:12:21.931 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: osdmap e86: 8 total, 8 up, 8 in 2024-03-24T12:12:21.932 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:21.932 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:21.932 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:21 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:21.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: pgmap v84: 876 pgs: 160 unknown, 241 creating+peering, 475 active+clean; 145 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 546 op/s 2024-03-24T12:12:21.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi006-82159-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi006-82159-7"}]': finished 2024-03-24T12:12:21.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi006-82972-1"}]': finished 2024-03-24T12:12:21.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3070372572' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi006-82972-1"}]: dispatch 2024-03-24T12:12:21.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:21.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: threexx 2024-03-24T12:12:21.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1967727185' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:21.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2845973739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:21.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: osdmap e85: 8 total, 8 up, 8 in 2024-03-24T12:12:21.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: Health check update: 17 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:21.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi006-82972-1"}]: dispatch 2024-03-24T12:12:21.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-24T12:12:21.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:21.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi006-82783-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi006-82783-2"}]': finished 2024-03-24T12:12:21.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi006-81813-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi006-81813-16"}]': finished 2024-03-24T12:12:21.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi006-81832-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi006-81832-22"}]': finished 2024-03-24T12:12:21.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi006-82140-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi006-82140-7"}]': finished 2024-03-24T12:12:21.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2547004740' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:21.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1967727185' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:21.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:21.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: pgmap v87: 892 pgs: 304 unknown, 145 creating+peering, 443 active+clean; 145 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 6.0 MiB/s rd, 25 MiB/s wr, 28 op/s 2024-03-24T12:12:21.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: osdmap e86: 8 total, 8 up, 8 in 2024-03-24T12:12:21.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:21.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:21.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:21.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: pgmap v84: 876 pgs: 160 unknown, 241 creating+peering, 475 active+clean; 145 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 546 op/s 2024-03-24T12:12:21.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi006-82159-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi006-82159-7"}]': finished 2024-03-24T12:12:21.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi006-82972-1"}]': finished 2024-03-24T12:12:21.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3070372572' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi006-82972-1"}]: dispatch 2024-03-24T12:12:21.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:21.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: threexx 2024-03-24T12:12:21.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1967727185' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:21.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2845973739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:21.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: osdmap e85: 8 total, 8 up, 8 in 2024-03-24T12:12:21.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: Health check update: 17 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:21.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi006-82972-1"}]: dispatch 2024-03-24T12:12:21.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-24T12:12:21.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:21.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi006-82783-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi006-82783-2"}]': finished 2024-03-24T12:12:21.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi006-81813-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi006-81813-16"}]': finished 2024-03-24T12:12:21.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi006-81832-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi006-81832-22"}]': finished 2024-03-24T12:12:21.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi006-82140-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi006-82140-7"}]': finished 2024-03-24T12:12:21.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2547004740' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:21.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1967727185' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:21.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:21.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: pgmap v87: 892 pgs: 304 unknown, 145 creating+peering, 443 active+clean; 145 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 6.0 MiB/s rd, 25 MiB/s wr, 28 op/s 2024-03-24T12:12:21.965 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: osdmap e86: 8 total, 8 up, 8 in 2024-03-24T12:12:21.965 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:21.965 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:21.965 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:21 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:22.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:12:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:12:23.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-03-24T12:12:23.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi006-82972-1"}]': finished 2024-03-24T12:12:23.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:23.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:23.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:23.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:22 smithi090 ceph-mon[33646]: osdmap e87: 8 total, 8 up, 8 in 2024-03-24T12:12:23.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1967727185' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi006-81895-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:23.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1149679251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:23.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2845973739' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-4", "tierpool": "test-rados-api-smithi006-82061-4-cache"}]: dispatch 2024-03-24T12:12:23.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:23.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:23.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi006-81895-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:23.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-4", "tierpool": "test-rados-api-smithi006-82061-4-cache"}]: dispatch 2024-03-24T12:12:23.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4079390308' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi006-82972-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:23.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/370521642' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:23.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi006-82972-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:23.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:23.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/370521642' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:23.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-03-24T12:12:23.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi006-82972-1"}]': finished 2024-03-24T12:12:23.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:23.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:23.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:23.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[30108]: osdmap e87: 8 total, 8 up, 8 in 2024-03-24T12:12:23.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1967727185' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi006-81895-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:23.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1149679251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:23.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2845973739' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-4", "tierpool": "test-rados-api-smithi006-82061-4-cache"}]: dispatch 2024-03-24T12:12:23.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:23.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:23.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi006-81895-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:23.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-4", "tierpool": "test-rados-api-smithi006-82061-4-cache"}]: dispatch 2024-03-24T12:12:23.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4079390308' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi006-82972-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:23.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/370521642' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:23.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi006-82972-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:23.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:23.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/370521642' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:23.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-03-24T12:12:23.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi006-82972-1"}]': finished 2024-03-24T12:12:23.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:23.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:23.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:23.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[41734]: osdmap e87: 8 total, 8 up, 8 in 2024-03-24T12:12:23.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1967727185' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi006-81895-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:23.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1149679251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:23.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2845973739' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-4", "tierpool": "test-rados-api-smithi006-82061-4-cache"}]: dispatch 2024-03-24T12:12:23.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:23.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:23.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi006-81895-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:23.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-4", "tierpool": "test-rados-api-smithi006-82061-4-cache"}]: dispatch 2024-03-24T12:12:23.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4079390308' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi006-82972-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:23.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/370521642' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:23.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi006-82972-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:23.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:23.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/370521642' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:23.286 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] cmd: Running main() from gmock_main.cc 2024-03-24T12:12:23.286 INFO:tasks.workunit.client.0.smithi006.stdout: cmd: [==========] Running 3 tests from 1 test suite. 2024-03-24T12:12:23.287 INFO:tasks.workunit.client.0.smithi006.stdout: cmd: [----------] Global test environment set-up. 2024-03-24T12:12:23.287 INFO:tasks.workunit.client.0.smithi006.stdout: cmd: [----------] 3 tests from NeoRadosCmd 2024-03-24T12:12:23.287 INFO:tasks.workunit.client.0.smithi006.stdout: cmd: [ RUN ] NeoRadosCmd.MonDescribe 2024-03-24T12:12:23.287 INFO:tasks.workunit.client.0.smithi006.stdout: cmd: [ OK ] NeoRadosCmd.MonDescribe (10697 ms) 2024-03-24T12:12:23.287 INFO:tasks.workunit.client.0.smithi006.stdout: cmd: [ RUN ] NeoRadosCmd.OSDCmd 2024-03-24T12:12:23.287 INFO:tasks.workunit.client.0.smithi006.stdout: cmd: [ OK ] NeoRadosCmd.OSDCmd (5750 ms) 2024-03-24T12:12:23.287 INFO:tasks.workunit.client.0.smithi006.stdout: cmd: [ RUN ] NeoRadosCmd.PGCmd 2024-03-24T12:12:23.287 INFO:tasks.workunit.client.0.smithi006.stdout: cmd: [ OK ] NeoRadosCmd.PGCmd (5081 ms) 2024-03-24T12:12:23.287 INFO:tasks.workunit.client.0.smithi006.stdout: cmd: [----------] 3 tests from NeoRadosCmd (21528 ms total) 2024-03-24T12:12:23.287 INFO:tasks.workunit.client.0.smithi006.stdout: cmd: 2024-03-24T12:12:23.287 INFO:tasks.workunit.client.0.smithi006.stdout: cmd: [----------] Global test environment tear-down 2024-03-24T12:12:23.287 INFO:tasks.workunit.client.0.smithi006.stdout: cmd: [==========] 3 tests from 1 test suite ran. (21528 ms total) 2024-03-24T12:12:23.287 INFO:tasks.workunit.client.0.smithi006.stdout: cmd: [ PASSED ] 3 tests. 2024-03-24T12:12:23.833 INFO:tasks.workunit.client.0.smithi006.stdout: watch_notify: Running main() from gmock_main.cc 2024-03-24T12:12:23.833 INFO:tasks.workunit.client.0.smithi006.stdout: watch_notify: [==========] Running 2 tests from 1 test suite. 2024-03-24T12:12:23.833 INFO:tasks.workunit.client.0.smithi006.stdout: watch_notify: [----------] Global test environment set-up. 2024-03-24T12:12:23.833 INFO:tasks.workunit.client.0.smithi006.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify 2024-03-24T12:12:23.833 INFO:tasks.workunit.client.0.smithi006.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotify 2024-03-24T12:12:23.833 INFO:tasks.workunit.client.0.smithi006.stdout: watch_notify: handle_notify cookie 94866273303264 notify_id 335007449088 notifier_gid 14904 2024-03-24T12:12:23.834 INFO:tasks.workunit.client.0.smithi006.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotify (11341 ms) 2024-03-24T12:12:23.834 INFO:tasks.workunit.client.0.smithi006.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-03-24T12:12:23.834 INFO:tasks.workunit.client.0.smithi006.stdout: watch_notify: Trying... 2024-03-24T12:12:23.834 INFO:tasks.workunit.client.0.smithi006.stdout: watch_notify: handle_notify cookie 94866286032656 notify_id 360777252867 notifier_gid 25130 2024-03-24T12:12:23.834 INFO:tasks.workunit.client.0.smithi006.stdout: watch_notify: Waiting for 3.000000000s 2024-03-24T12:12:23.834 INFO:tasks.workunit.client.0.smithi006.stdout: watch_notify: Timed out. 2024-03-24T12:12:23.834 INFO:tasks.workunit.client.0.smithi006.stdout: watch_notify: Flushing... 2024-03-24T12:12:23.834 INFO:tasks.workunit.client.0.smithi006.stdout: watch_notify: Flushed... 2024-03-24T12:12:23.834 INFO:tasks.workunit.client.0.smithi006.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (10683 ms) 2024-03-24T12:12:23.834 INFO:tasks.workunit.client.0.smithi006.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify (22024 ms total) 2024-03-24T12:12:23.834 INFO:tasks.workunit.client.0.smithi006.stdout: watch_notify: 2024-03-24T12:12:23.834 INFO:tasks.workunit.client.0.smithi006.stdout: watch_notify: [----------] Global test environment tear-down 2024-03-24T12:12:23.834 INFO:tasks.workunit.client.0.smithi006.stdout: watch_notify: [==========] 2 tests from 1 test suite ran. (22024 ms total) 2024-03-24T12:12:23.834 INFO:tasks.workunit.client.0.smithi006.stdout: watch_notify: [ PASSED ] 2 tests. 2024-03-24T12:12:24.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:23 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:12:24.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:23 smithi090 ceph-mon[33646]: fourxx 2024-03-24T12:12:24.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:24.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-03-24T12:12:24.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:24.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]': finished 2024-03-24T12:12:24.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-4", "tierpool": "test-rados-api-smithi006-82061-4-cache"}]': finished 2024-03-24T12:12:24.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi006-82972-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:24.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1149679251' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:24.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2845973739' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-4", "overlaypool": "test-rados-api-smithi006-82061-4-cache"}]: dispatch 2024-03-24T12:12:24.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4079390308' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi006-82972-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi006-82972-2"}]: dispatch 2024-03-24T12:12:24.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3587505853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi006-81775-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:24.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/370521642' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:24.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:23 smithi090 ceph-mon[33646]: osdmap e88: 8 total, 8 up, 8 in 2024-03-24T12:12:24.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi006-82972-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi006-82972-2"}]: dispatch 2024-03-24T12:12:24.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi006-81775-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:24.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:24.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:24.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-4", "overlaypool": "test-rados-api-smithi006-82061-4-cache"}]: dispatch 2024-03-24T12:12:24.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:12:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[30108]: fourxx 2024-03-24T12:12:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-03-24T12:12:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]': finished 2024-03-24T12:12:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-4", "tierpool": "test-rados-api-smithi006-82061-4-cache"}]': finished 2024-03-24T12:12:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi006-82972-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1149679251' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:24.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2845973739' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-4", "overlaypool": "test-rados-api-smithi006-82061-4-cache"}]: dispatch 2024-03-24T12:12:24.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4079390308' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi006-82972-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi006-82972-2"}]: dispatch 2024-03-24T12:12:24.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3587505853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi006-81775-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:24.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/370521642' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:24.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[30108]: osdmap e88: 8 total, 8 up, 8 in 2024-03-24T12:12:24.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi006-82972-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi006-82972-2"}]: dispatch 2024-03-24T12:12:24.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi006-81775-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:24.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:24.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:24.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-4", "overlaypool": "test-rados-api-smithi006-82061-4-cache"}]: dispatch 2024-03-24T12:12:24.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:24.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:12:24.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[41734]: fourxx 2024-03-24T12:12:24.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:24.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-03-24T12:12:24.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:24.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi006-82271-1"}]': finished 2024-03-24T12:12:24.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-4", "tierpool": "test-rados-api-smithi006-82061-4-cache"}]': finished 2024-03-24T12:12:24.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi006-82972-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:24.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1149679251' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:24.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2845973739' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-4", "overlaypool": "test-rados-api-smithi006-82061-4-cache"}]: dispatch 2024-03-24T12:12:24.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4079390308' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi006-82972-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi006-82972-2"}]: dispatch 2024-03-24T12:12:24.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3587505853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi006-81775-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:24.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/370521642' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:24.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[41734]: osdmap e88: 8 total, 8 up, 8 in 2024-03-24T12:12:24.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi006-82972-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi006-82972-2"}]: dispatch 2024-03-24T12:12:24.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi006-81775-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:24.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:24.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi006-82271-1"}]: dispatch 2024-03-24T12:12:24.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-4", "overlaypool": "test-rados-api-smithi006-82061-4-cache"}]: dispatch 2024-03-24T12:12:24.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:25.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: pgmap v90: 820 pgs: 64 creating+peering, 128 unknown, 628 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 27 KiB/s rd, 30 KiB/s wr, 29 op/s 2024-03-24T12:12:25.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-03-24T12:12:25.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi006-81895-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi006-81895-10"}]': finished 2024-03-24T12:12:25.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi006-81775-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:25.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:25.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi006-82271-1"}]': finished 2024-03-24T12:12:25.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-4", "overlaypool": "test-rados-api-smithi006-82061-4-cache"}]': finished 2024-03-24T12:12:25.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2845973739' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-4-cache", "mode": "writeback"}]: dispatch 2024-03-24T12:12:25.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4236440874' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:25.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/370521642' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi006-82180-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:25.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3974774221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi006-82783-2"}]: dispatch 2024-03-24T12:12:25.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: osdmap e89: 8 total, 8 up, 8 in 2024-03-24T12:12:25.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3591427439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:25.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3619165262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:25.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4087805071' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:25.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2158124404' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:25.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-4-cache", "mode": "writeback"}]: dispatch 2024-03-24T12:12:25.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi006-82180-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:25.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi006-82783-2"}]: dispatch 2024-03-24T12:12:25.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:25.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi006-82159-7"}]: dispatch 2024-03-24T12:12:25.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:25.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:25.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:25.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:25.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:25.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: pgmap v90: 820 pgs: 64 creating+peering, 128 unknown, 628 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 27 KiB/s rd, 30 KiB/s wr, 29 op/s 2024-03-24T12:12:25.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-03-24T12:12:25.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi006-81895-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi006-81895-10"}]': finished 2024-03-24T12:12:25.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi006-81775-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:25.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:25.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi006-82271-1"}]': finished 2024-03-24T12:12:25.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-4", "overlaypool": "test-rados-api-smithi006-82061-4-cache"}]': finished 2024-03-24T12:12:25.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2845973739' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-4-cache", "mode": "writeback"}]: dispatch 2024-03-24T12:12:25.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4236440874' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:25.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/370521642' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi006-82180-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:25.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3974774221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi006-82783-2"}]: dispatch 2024-03-24T12:12:25.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: osdmap e89: 8 total, 8 up, 8 in 2024-03-24T12:12:25.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3591427439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:25.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3619165262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:25.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4087805071' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:25.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2158124404' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:25.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-4-cache", "mode": "writeback"}]: dispatch 2024-03-24T12:12:25.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi006-82180-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:25.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: pgmap v90: 820 pgs: 64 creating+peering, 128 unknown, 628 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 27 KiB/s rd, 30 KiB/s wr, 29 op/s 2024-03-24T12:12:25.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4197379591' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-03-24T12:12:25.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi006-81895-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi006-81895-10"}]': finished 2024-03-24T12:12:25.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi006-81775-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:25.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:25.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi006-82271-1"}]': finished 2024-03-24T12:12:25.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-4", "overlaypool": "test-rados-api-smithi006-82061-4-cache"}]': finished 2024-03-24T12:12:25.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2845973739' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-4-cache", "mode": "writeback"}]: dispatch 2024-03-24T12:12:25.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4236440874' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:25.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/370521642' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi006-82180-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:25.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3974774221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi006-82783-2"}]: dispatch 2024-03-24T12:12:25.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: osdmap e89: 8 total, 8 up, 8 in 2024-03-24T12:12:25.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3591427439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:25.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3619165262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:25.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4087805071' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:25.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2158124404' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:25.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-4-cache", "mode": "writeback"}]: dispatch 2024-03-24T12:12:25.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi006-82180-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:25.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi006-82783-2"}]: dispatch 2024-03-24T12:12:25.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:25.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi006-82159-7"}]: dispatch 2024-03-24T12:12:25.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:25.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:25.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:25.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:25.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:25.215 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi006-82783-2"}]: dispatch 2024-03-24T12:12:25.215 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:25.215 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi006-82159-7"}]: dispatch 2024-03-24T12:12:25.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:25.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:25.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:25.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:25.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:25.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:12:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:12:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:12:25.758 INFO:tasks.workunit.client.0.smithi006.stdout: ec_io: Running main() from gmock_main.cc 2024-03-24T12:12:25.758 INFO:tasks.workunit.client.0.smithi006.stdout: ec_io: [==========] Running 2 tests from 1 test suite. 2024-03-24T12:12:25.758 INFO:tasks.workunit.client.0.smithi006.stdout: ec_io: [----------] Global test environment set-up. 2024-03-24T12:12:25.759 INFO:tasks.workunit.client.0.smithi006.stdout: ec_io: [----------] 2 tests from NeoRadosECIo 2024-03-24T12:12:25.759 INFO:tasks.workunit.client.0.smithi006.stdout: ec_io: [ RUN ] NeoRadosECIo.SimpleWrite 2024-03-24T12:12:25.759 INFO:tasks.workunit.client.0.smithi006.stdout: ec_io: [ OK ] NeoRadosECIo.SimpleWrite (15474 ms) 2024-03-24T12:12:25.759 INFO:tasks.workunit.client.0.smithi006.stdout: ec_io: [ RUN ] NeoRadosECIo.ReadOp 2024-03-24T12:12:25.759 INFO:tasks.workunit.client.0.smithi006.stdout: ec_io: [ OK ] NeoRadosECIo.ReadOp (8568 ms) 2024-03-24T12:12:25.759 INFO:tasks.workunit.client.0.smithi006.stdout: ec_io: [----------] 2 tests from NeoRadosECIo (24042 ms total) 2024-03-24T12:12:25.759 INFO:tasks.workunit.client.0.smithi006.stdout: ec_io: 2024-03-24T12:12:25.759 INFO:tasks.workunit.client.0.smithi006.stdout: ec_io: [----------] Global test environment tear-down 2024-03-24T12:12:25.759 INFO:tasks.workunit.client.0.smithi006.stdout: ec_io: [==========] 2 tests from 1 test suite ran. (24042 ms total) 2024-03-24T12:12:25.759 INFO:tasks.workunit.client.0.smithi006.stdout: ec_io: [ PASSED ] 2 tests. 2024-03-24T12:12:25.764 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: Running main() from gmock_main.cc 2024-03-24T12:12:25.818 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-03-24T12:12:25.818 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [----------] Global test environment set-up. 2024-03-24T12:12:25.818 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-03-24T12:12:25.818 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-03-24T12:12:25.818 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (81 ms) 2024-03-24T12:12:25.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-03-24T12:12:25.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-03-24T12:12:25.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-03-24T12:12:25.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: no timeout :/ 2024-03-24T12:12:25.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: no timeout :/ 2024-03-24T12:12:25.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: no timeout :/ 2024-03-24T12:12:25.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: no timeout :/ 2024-03-24T12:12:25.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: no timeout :/ 2024-03-24T12:12:25.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (28 ms) 2024-03-24T12:12:25.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-03-24T12:12:25.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (9 ms) 2024-03-24T12:12:25.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-03-24T12:12:25.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIo.Checksum (3 ms) 2024-03-24T12:12:25.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-03-24T12:12:25.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (6 ms) 2024-03-24T12:12:25.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-03-24T12:12:25.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (2 ms) 2024-03-24T12:12:25.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-03-24T12:12:25.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (5 ms) 2024-03-24T12:12:25.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-03-24T12:12:25.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (2 ms) 2024-03-24T12:12:25.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-03-24T12:12:25.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIo.TruncTest (3 ms) 2024-03-24T12:12:25.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-03-24T12:12:25.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (2 ms) 2024-03-24T12:12:25.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-03-24T12:12:25.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (3 ms) 2024-03-24T12:12:25.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-03-24T12:12:25.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIo.RmXattr (7 ms) 2024-03-24T12:12:25.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-03-24T12:12:25.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIo.XattrIter (4 ms) 2024-03-24T12:12:25.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [----------] 14 tests from LibRadosIo (155 ms total) 2024-03-24T12:12:25.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: 2024-03-24T12:12:25.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-03-24T12:12:25.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-03-24T12:12:25.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (1185 ms) 2024-03-24T12:12:25.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-03-24T12:12:25.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (6 ms) 2024-03-24T12:12:25.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-03-24T12:12:25.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (6 ms) 2024-03-24T12:12:25.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-03-24T12:12:25.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (5 ms) 2024-03-24T12:12:25.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-03-24T12:12:25.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (10 ms) 2024-03-24T12:12:25.822 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-03-24T12:12:25.822 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (5 ms) 2024-03-24T12:12:25.822 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-03-24T12:12:25.822 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (3 ms) 2024-03-24T12:12:25.822 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-03-24T12:12:25.822 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (4 ms) 2024-03-24T12:12:25.822 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-03-24T12:12:25.822 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (9 ms) 2024-03-24T12:12:25.822 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-03-24T12:12:25.822 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (5 ms) 2024-03-24T12:12:25.822 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [----------] 10 tests from LibRadosIoEC (1238 ms total) 2024-03-24T12:12:25.822 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: 2024-03-24T12:12:25.822 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [----------] Global test environment tear-down 2024-03-24T12:12:25.822 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [==========] 24 tests from 2 test suites ran. (24392 ms total) 2024-03-24T12:12:25.823 INFO:tasks.workunit.client.0.smithi006.stdout: api_io: [ PASSED ] 24 tests. 2024-03-24T12:12:25.823 INFO:tasks.workunit.client.0.smithi006.stdout: list: Running main() from gmock_main.cc 2024-03-24T12:12:25.825 INFO:tasks.workunit.client.0.smithi006.stdout: list: [==========] Running 3 tests from 1 test suite. 2024-03-24T12:12:25.825 INFO:tasks.workunit.client.0.smithi006.stdout: list: [----------] Global test environment set-up. 2024-03-24T12:12:25.825 INFO:tasks.workunit.client.0.smithi006.stdout: list: [----------] 3 tests from NeoradosList 2024-03-24T12:12:25.825 INFO:tasks.workunit.client.0.smithi006.stdout: list: [ RUN ] NeoradosList.ListObjects 2024-03-24T12:12:25.825 INFO:tasks.workunit.client.0.smithi006.stdout: list: [ OK ] NeoradosList.ListObjects (11433 ms) 2024-03-24T12:12:25.825 INFO:tasks.workunit.client.0.smithi006.stdout: list: [ RUN ] NeoradosList.ListObjectsNS 2024-03-24T12:12:25.825 INFO:tasks.workunit.client.0.smithi006.stdout: list: [ OK ] NeoradosList.ListObjectsNS (8479 ms) 2024-03-24T12:12:25.825 INFO:tasks.workunit.client.0.smithi006.stdout: list: [ RUN ] NeoradosList.ListObjectsMany 2024-03-24T12:12:25.825 INFO:tasks.workunit.client.0.smithi006.stdout: list: [ OK ] NeoradosList.ListObjectsMany (4132 ms) 2024-03-24T12:12:25.825 INFO:tasks.workunit.client.0.smithi006.stdout: list: [----------] 3 tests from NeoradosList (24044 ms total) 2024-03-24T12:12:25.825 INFO:tasks.workunit.client.0.smithi006.stdout: list: 2024-03-24T12:12:25.825 INFO:tasks.workunit.client.0.smithi006.stdout: list: [----------] Global test environment tear-down 2024-03-24T12:12:25.825 INFO:tasks.workunit.client.0.smithi006.stdout: list: [==========] 3 tests from 1 test suite ran. (24044 ms total) 2024-03-24T12:12:25.826 INFO:tasks.workunit.client.0.smithi006.stdout: list: [ PASSED ] 3 tests. 2024-03-24T12:12:25.826 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: Running main() from gmock_main.cc 2024-03-24T12:12:25.826 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2024-03-24T12:12:25.826 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [----------] Global test environment set-up. 2024-03-24T12:12:25.826 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [----------] 5 tests from LibRadosStat 2024-03-24T12:12:25.826 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-03-24T12:12:25.826 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [ OK ] LibRadosStat.Stat (1272 ms) 2024-03-24T12:12:25.826 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2024-03-24T12:12:25.826 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (38 ms) 2024-03-24T12:12:25.826 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-03-24T12:12:25.826 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [ OK ] LibRadosStat.StatNS (18 ms) 2024-03-24T12:12:25.826 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-03-24T12:12:25.826 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (0 ms) 2024-03-24T12:12:25.826 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-03-24T12:12:25.826 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (4 ms) 2024-03-24T12:12:25.827 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [----------] 5 tests from LibRadosStat (1332 ms total) 2024-03-24T12:12:25.827 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: 2024-03-24T12:12:25.827 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-03-24T12:12:25.827 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-03-24T12:12:25.827 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1165 ms) 2024-03-24T12:12:25.827 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-03-24T12:12:25.827 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (23 ms) 2024-03-24T12:12:25.827 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-03-24T12:12:25.827 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (0 ms) 2024-03-24T12:12:25.827 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-03-24T12:12:25.827 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (3 ms) 2024-03-24T12:12:25.827 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (1191 ms total) 2024-03-24T12:12:25.827 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: 2024-03-24T12:12:25.827 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [----------] Global test environment tear-down 2024-03-24T12:12:25.827 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (24290 ms total) 2024-03-24T12:12:25.828 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat: [ PASSED ] 9 tests. 2024-03-24T12:12:25.828 INFO:tasks.workunit.client.0.smithi006.stdout: LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-03-24T12:12:25.828 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (4 ms) 2024-03-24T12:12:25.828 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-03-24T12:12:25.828 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (5 ms) 2024-03-24T12:12:25.828 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-03-24T12:12:25.828 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (3 ms) 2024-03-24T12:12:25.828 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-03-24T12:12:25.828 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (4 ms) 2024-03-24T12:12:25.828 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-03-24T12:12:25.828 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (3 ms) 2024-03-24T12:12:25.828 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-03-24T12:12:25.828 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (3 ms) 2024-03-24T12:12:25.828 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-03-24T12:12:25.828 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (4 ms) 2024-03-24T12:12:25.828 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-03-24T12:12:25.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (9 ms) 2024-03-24T12:12:25.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-03-24T12:12:25.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (4 ms) 2024-03-24T12:12:25.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-03-24T12:12:25.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (7 ms) 2024-03-24T12:12:25.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-03-24T12:12:25.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (5 ms) 2024-03-24T12:12:25.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-03-24T12:12:25.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (6 ms) 2024-03-24T12:12:25.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (1262 ms total) 2024-03-24T12:12:25.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: 2024-03-24T12:12:25.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [----------] Global test environment tear-down 2024-03-24T12:12:25.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (24395 ms total) 2024-03-24T12:12:25.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-03-24T12:12:25.830 INFO:tasks.workunit.client.0.smithi006.stdout:.c [INF] from='client.? 172.21.15.6:0/1967727185' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:25.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:21.143823+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:25.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:21.144229+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:25.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: got: 2024-03-24T12:12:21.418528+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:25.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (12667 ms) 2024-03-24T12:12:25.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (24270 ms total) 2024-03-24T12:12:25.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: 2024-03-24T12:12:25.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: [----------] Global test environment tear-down 2024-03-24T12:12:25.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (24271 ms total) 2024-03-24T12:12:25.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_cmd: [ PASSED ] 4 tests. 2024-03-24T12:12:26.045 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-03-24T12:12:26.046 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-03-24T12:12:26.046 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-03-24T12:12:26.046 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-03-24T12:12:26.046 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: seed 82159 2024-03-24T12:12:26.046 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-03-24T12:12:26.046 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (308 ms) 2024-03-24T12:12:26.046 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-03-24T12:12:26.046 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (6 ms) 2024-03-24T12:12:26.046 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-03-24T12:12:26.046 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (0 ms) 2024-03-24T12:12:26.046 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-03-24T12:12:26.046 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (1 ms) 2024-03-24T12:12:26.046 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-03-24T12:12:26.047 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (45 ms) 2024-03-24T12:12:26.047 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (360 ms total) 2024-03-24T12:12:26.047 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: 2024-03-24T12:12:26.047 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-03-24T12:12:26.047 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-03-24T12:12:26.047 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (1184 ms) 2024-03-24T12:12:26.047 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-03-24T12:12:26.047 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (199 ms) 2024-03-24T12:12:26.047 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-03-24T12:12:26.047 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (503 ms) 2024-03-24T12:12:26.047 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-03-24T12:12:26.047 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (6 ms) 2024-03-24T12:12:26.047 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (1892 ms total) 2024-03-24T12:12:26.047 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: 2024-03-24T12:12:26.047 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-03-24T12:12:26.047 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (24555 ms total) 2024-03-24T12:12:26.048 INFO:tasks.workunit.client.0.smithi006.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-03-24T12:12:26.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:12:26.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi006-82972-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi006-82972-2"}]': finished 2024-03-24T12:12:26.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-4-cache", "mode": "writeback"}]': finished 2024-03-24T12:12:26.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi006-82783-2"}]': finished 2024-03-24T12:12:26.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:26.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi006-82159-7"}]': finished 2024-03-24T12:12:26.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:26.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi006-82140-7"}]': finished 2024-03-24T12:12:26.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi006-81813-16"}]': finished 2024-03-24T12:12:26.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22"}]': finished 2024-03-24T12:12:26.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3619165262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:26.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4087805071' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:26.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3591427439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:26.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3974774221' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi006-82783-2"}]: dispatch 2024-03-24T12:12:26.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2845973739' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-4"}]: dispatch 2024-03-24T12:12:26.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: pgmap v93: 724 pgs: 32 creating+peering, 208 unknown, 484 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:12:26.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: osdmap e90: 8 total, 8 up, 8 in 2024-03-24T12:12:26.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3543056798' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi006-82076-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:26.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:26.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2356903427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:26.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi006-82159-7"}]: dispatch 2024-03-24T12:12:26.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:26.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi006-82783-2"}]: dispatch 2024-03-24T12:12:26.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:26.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:26.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:26.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:26.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:26.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: pool 'PoolQuotaPP_smithi006-81794-3' is full (reached quota's max_bytes: 4 KiB) 2024-03-24T12:12:26.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-24T12:12:26.431 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi006-82180-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]': finished 2024-03-24T12:12:26.431 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi006-82159-7"}]': finished 2024-03-24T12:12:26.431 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi006-81813-16"}]': finished 2024-03-24T12:12:26.431 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi006-82783-2"}]': finished 2024-03-24T12:12:26.431 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi006-81832-22"}]': finished 2024-03-24T12:12:26.431 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi006-82140-7"}]': finished 2024-03-24T12:12:26.431 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:26.431 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi006-81943-10"}]': finished 2024-03-24T12:12:26.431 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: osdmap e91: 8 total, 8 up, 8 in 2024-03-24T12:12:26.431 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2356903427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:12:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi006-82972-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi006-82972-2"}]': finished 2024-03-24T12:12:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-4-cache", "mode": "writeback"}]': finished 2024-03-24T12:12:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi006-82783-2"}]': finished 2024-03-24T12:12:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi006-82159-7"}]': finished 2024-03-24T12:12:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi006-82140-7"}]': finished 2024-03-24T12:12:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi006-81813-16"}]': finished 2024-03-24T12:12:26.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22"}]': finished 2024-03-24T12:12:26.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3619165262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:26.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4087805071' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:26.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3591427439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:26.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3974774221' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi006-82783-2"}]: dispatch 2024-03-24T12:12:26.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2845973739' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-4"}]: dispatch 2024-03-24T12:12:26.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: pgmap v93: 724 pgs: 32 creating+peering, 208 unknown, 484 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:12:26.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: osdmap e90: 8 total, 8 up, 8 in 2024-03-24T12:12:26.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3543056798' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi006-82076-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:26.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:26.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:12:26.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi006-82972-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi006-82972-2"}]': finished 2024-03-24T12:12:26.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-4-cache", "mode": "writeback"}]': finished 2024-03-24T12:12:26.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi006-82783-2"}]': finished 2024-03-24T12:12:26.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:26.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi006-82159-7"}]': finished 2024-03-24T12:12:26.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:26.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi006-82140-7"}]': finished 2024-03-24T12:12:26.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi006-81813-16"}]': finished 2024-03-24T12:12:26.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi006-81832-22"}]': finished 2024-03-24T12:12:26.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3619165262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:26.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4087805071' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:26.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3591427439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:26.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3974774221' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi006-82783-2"}]: dispatch 2024-03-24T12:12:26.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2845973739' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-4"}]: dispatch 2024-03-24T12:12:26.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: pgmap v93: 724 pgs: 32 creating+peering, 208 unknown, 484 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:12:26.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: osdmap e90: 8 total, 8 up, 8 in 2024-03-24T12:12:26.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3543056798' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi006-82076-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:26.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:26.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2356903427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:26.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi006-82159-7"}]: dispatch 2024-03-24T12:12:26.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:26.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi006-82783-2"}]: dispatch 2024-03-24T12:12:26.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:26.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:26.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:26.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:26.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:26.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: pool 'PoolQuotaPP_smithi006-81794-3' is full (reached quota's max_bytes: 4 KiB) 2024-03-24T12:12:26.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-24T12:12:26.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi006-82180-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]': finished 2024-03-24T12:12:26.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi006-82159-7"}]': finished 2024-03-24T12:12:26.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi006-81813-16"}]': finished 2024-03-24T12:12:26.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi006-82783-2"}]': finished 2024-03-24T12:12:26.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi006-81832-22"}]': finished 2024-03-24T12:12:26.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi006-82140-7"}]': finished 2024-03-24T12:12:26.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:26.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi006-81943-10"}]': finished 2024-03-24T12:12:26.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: osdmap e91: 8 total, 8 up, 8 in 2024-03-24T12:12:26.465 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2356903427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:26.466 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2356903427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:26.466 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi006-82159-7"}]: dispatch 2024-03-24T12:12:26.466 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi006-81813-16"}]: dispatch 2024-03-24T12:12:26.466 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi006-82783-2"}]: dispatch 2024-03-24T12:12:26.466 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi006-81832-22"}]: dispatch 2024-03-24T12:12:26.466 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi006-82140-7"}]: dispatch 2024-03-24T12:12:26.466 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:26.466 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:26.466 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:26.466 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: pool 'PoolQuotaPP_smithi006-81794-3' is full (reached quota's max_bytes: 4 KiB) 2024-03-24T12:12:26.466 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-24T12:12:26.466 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi006-82180-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]': finished 2024-03-24T12:12:26.467 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1540360223' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi006-82159-7"}]': finished 2024-03-24T12:12:26.467 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi006-81813-16"}]': finished 2024-03-24T12:12:26.467 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi006-82783-2"}]': finished 2024-03-24T12:12:26.467 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi006-81832-22"}]': finished 2024-03-24T12:12:26.467 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi006-82140-7"}]': finished 2024-03-24T12:12:26.467 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:26.467 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi006-81943-10"}]': finished 2024-03-24T12:12:26.467 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: osdmap e91: 8 total, 8 up, 8 in 2024-03-24T12:12:26.467 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2356903427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:26.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: Running main() from gmock_main.cc 2024-03-24T12:12:26.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-03-24T12:12:26.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [----------] Global test environment set-up. 2024-03-24T12:12:26.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-03-24T12:12:26.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-03-24T12:12:26.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (1128 ms) 2024-03-24T12:12:26.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-03-24T12:12:26.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ OK ] LibRadosLock.LockShared (4 ms) 2024-03-24T12:12:26.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-03-24T12:12:26.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1175 ms) 2024-03-24T12:12:26.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-03-24T12:12:26.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1028 ms) 2024-03-24T12:12:26.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-03-24T12:12:26.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (4 ms) 2024-03-24T12:12:26.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-03-24T12:12:26.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ OK ] LibRadosLock.Unlock (3 ms) 2024-03-24T12:12:26.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-03-24T12:12:26.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (5 ms) 2024-03-24T12:12:26.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-03-24T12:12:26.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (3 ms) 2024-03-24T12:12:26.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [----------] 8 tests from LibRadosLock (3350 ms total) 2024-03-24T12:12:26.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: 2024-03-24T12:12:26.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-03-24T12:12:26.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-03-24T12:12:26.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (2491 ms) 2024-03-24T12:12:26.835 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-03-24T12:12:26.835 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (9 ms) 2024-03-24T12:12:26.835 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-03-24T12:12:26.835 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1032 ms) 2024-03-24T12:12:26.835 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-03-24T12:12:26.835 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1103 ms) 2024-03-24T12:12:26.835 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-03-24T12:12:26.835 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (18 ms) 2024-03-24T12:12:26.835 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-03-24T12:12:26.835 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (5 ms) 2024-03-24T12:12:26.835 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-03-24T12:12:26.835 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (6 ms) 2024-03-24T12:12:26.835 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-03-24T12:12:26.835 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (3 ms) 2024-03-24T12:12:26.835 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (4667 ms total) 2024-03-24T12:12:26.836 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: 2024-03-24T12:12:26.836 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [----------] Global test environment tear-down 2024-03-24T12:12:26.836 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (25407 ms total) 2024-03-24T12:12:26.836 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock: [ PASSED ] 16 tests. 2024-03-24T12:12:27.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:27 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:27.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:27 smithi090 ceph-mon[33646]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:27.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:27 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-4"}]: dispatch 2024-03-24T12:12:27.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:27 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi006-82076-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:27.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:27 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:27.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:27 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:27.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:27 smithi006 ceph-mon[30108]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:27.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:27 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-4"}]: dispatch 2024-03-24T12:12:27.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:27 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi006-82076-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:27.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:27 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:27.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:27 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi006-81943-10"}]: dispatch 2024-03-24T12:12:27.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:27 smithi006 ceph-mon[41734]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:27.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:27 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-4"}]: dispatch 2024-03-24T12:12:27.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:27 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi006-82076-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:27.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:27 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi006-81943-10"}]': finished 2024-03-24T12:12:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-4"}]': finished 2024-03-24T12:12:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi006-82076-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2845973739' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-4", "tierpool": "test-rados-api-smithi006-82061-4-cache"}]: dispatch 2024-03-24T12:12:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/955250561' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi006-81775-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82271-6", "pg_num": 4}]: dispatch 2024-03-24T12:12:28.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: pgmap v96: 764 pgs: 19 creating+peering, 225 unknown, 520 active+clean; 145 MiB data, 743 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:12:28.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: osdmap e92: 8 total, 8 up, 8 in 2024-03-24T12:12:28.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi006-81775-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:28.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82271-6", "pg_num": 4}]: dispatch 2024-03-24T12:12:28.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:28.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-4", "tierpool": "test-rados-api-smithi006-82061-4-cache"}]: dispatch 2024-03-24T12:12:28.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi006-81775-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:28.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82271-6", "pg_num": 4}]': finished 2024-03-24T12:12:28.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: osdmap e93: 8 total, 8 up, 8 in 2024-03-24T12:12:28.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1967727185' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:28.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/768703894' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:28.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4079390308' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi006-82972-2"}]: dispatch 2024-03-24T12:12:28.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "tierpool": "test-rados-api-smithi006-82271-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:28.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:28.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/355171516' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:28.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:28.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi006-82972-2"}]: dispatch 2024-03-24T12:12:28.714 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "tierpool": "test-rados-api-smithi006-82271-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:28.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi006-81943-10"}]': finished 2024-03-24T12:12:28.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-4"}]': finished 2024-03-24T12:12:28.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi006-82076-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:28.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2845973739' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-4", "tierpool": "test-rados-api-smithi006-82061-4-cache"}]: dispatch 2024-03-24T12:12:28.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/955250561' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi006-81775-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:28.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82271-6", "pg_num": 4}]: dispatch 2024-03-24T12:12:28.715 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: pgmap v96: 764 pgs: 19 creating+peering, 225 unknown, 520 active+clean; 145 MiB data, 743 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:12:28.715 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: osdmap e92: 8 total, 8 up, 8 in 2024-03-24T12:12:28.715 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi006-81775-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:28.715 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82271-6", "pg_num": 4}]: dispatch 2024-03-24T12:12:28.715 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:28.715 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-4", "tierpool": "test-rados-api-smithi006-82061-4-cache"}]: dispatch 2024-03-24T12:12:28.715 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi006-81775-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:28.715 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82271-6", "pg_num": 4}]': finished 2024-03-24T12:12:28.715 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: osdmap e93: 8 total, 8 up, 8 in 2024-03-24T12:12:28.715 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1967727185' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:28.715 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/768703894' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:28.715 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4079390308' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi006-82972-2"}]: dispatch 2024-03-24T12:12:28.715 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "tierpool": "test-rados-api-smithi006-82271-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:28.716 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:28.716 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/355171516' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:28.716 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:28.716 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi006-82972-2"}]: dispatch 2024-03-24T12:12:28.716 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "tierpool": "test-rados-api-smithi006-82271-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:28.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi006-81943-10"}]': finished 2024-03-24T12:12:28.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-4"}]': finished 2024-03-24T12:12:28.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi006-82076-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:28.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2845973739' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-4", "tierpool": "test-rados-api-smithi006-82061-4-cache"}]: dispatch 2024-03-24T12:12:28.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/955250561' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi006-81775-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:28.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82271-6", "pg_num": 4}]: dispatch 2024-03-24T12:12:28.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: pgmap v96: 764 pgs: 19 creating+peering, 225 unknown, 520 active+clean; 145 MiB data, 743 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:12:28.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: osdmap e92: 8 total, 8 up, 8 in 2024-03-24T12:12:28.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi006-81775-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:28.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82271-6", "pg_num": 4}]: dispatch 2024-03-24T12:12:28.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:28.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-4", "tierpool": "test-rados-api-smithi006-82061-4-cache"}]: dispatch 2024-03-24T12:12:28.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi006-81775-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:28.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82271-6", "pg_num": 4}]': finished 2024-03-24T12:12:28.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: osdmap e93: 8 total, 8 up, 8 in 2024-03-24T12:12:28.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1967727185' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:28.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/768703894' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:28.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4079390308' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi006-82972-2"}]: dispatch 2024-03-24T12:12:28.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "tierpool": "test-rados-api-smithi006-82271-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:28.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:28.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/355171516' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:28.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:28.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi006-82972-2"}]: dispatch 2024-03-24T12:12:28.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "tierpool": "test-rados-api-smithi006-82271-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:29.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:29.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:12:29.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-4", "tierpool": "test-rados-api-smithi006-82061-4-cache"}]': finished 2024-03-24T12:12:29.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10"}]': finished 2024-03-24T12:12:29.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/355171516' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:29.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:29.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi006-82972-2"}]': finished 2024-03-24T12:12:29.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "tierpool": "test-rados-api-smithi006-82271-6", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:12:29.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[30108]: osdmap e94: 8 total, 8 up, 8 in 2024-03-24T12:12:29.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4079390308' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi006-82972-2"}]: dispatch 2024-03-24T12:12:29.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "overlaypool": "test-rados-api-smithi006-82271-6"}]: dispatch 2024-03-24T12:12:29.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1967727185' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:29.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/370521642' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:29.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi006-82972-2"}]: dispatch 2024-03-24T12:12:29.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "overlaypool": "test-rados-api-smithi006-82271-6"}]: dispatch 2024-03-24T12:12:29.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:29.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:29.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:29.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:12:29.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-4", "tierpool": "test-rados-api-smithi006-82061-4-cache"}]': finished 2024-03-24T12:12:29.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10"}]': finished 2024-03-24T12:12:29.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/355171516' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:29.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:29.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi006-82972-2"}]': finished 2024-03-24T12:12:29.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "tierpool": "test-rados-api-smithi006-82271-6", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:12:29.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[41734]: osdmap e94: 8 total, 8 up, 8 in 2024-03-24T12:12:29.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4079390308' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi006-82972-2"}]: dispatch 2024-03-24T12:12:29.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "overlaypool": "test-rados-api-smithi006-82271-6"}]: dispatch 2024-03-24T12:12:29.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1967727185' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:29.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/370521642' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:29.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi006-82972-2"}]: dispatch 2024-03-24T12:12:29.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "overlaypool": "test-rados-api-smithi006-82271-6"}]: dispatch 2024-03-24T12:12:29.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:29.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:29.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:29 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:29.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:29 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:12:29.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-4", "tierpool": "test-rados-api-smithi006-82061-4-cache"}]': finished 2024-03-24T12:12:29.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi006-81895-10"}]': finished 2024-03-24T12:12:29.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:29 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/355171516' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:29.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:29.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi006-82972-2"}]': finished 2024-03-24T12:12:29.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "tierpool": "test-rados-api-smithi006-82271-6", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:12:29.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:29 smithi090 ceph-mon[33646]: osdmap e94: 8 total, 8 up, 8 in 2024-03-24T12:12:29.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:29 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4079390308' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi006-82972-2"}]: dispatch 2024-03-24T12:12:29.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:29 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "overlaypool": "test-rados-api-smithi006-82271-6"}]: dispatch 2024-03-24T12:12:29.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:29 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1967727185' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:29.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:29 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/370521642' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:29.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi006-82972-2"}]: dispatch 2024-03-24T12:12:29.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "overlaypool": "test-rados-api-smithi006-82271-6"}]: dispatch 2024-03-24T12:12:29.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi006-81895-10"}]: dispatch 2024-03-24T12:12:29.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:29.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-03-24T12:12:29.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-03-24T12:12:29.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-03-24T12:12:29.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-03-24T12:12:29.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: seed 81895 2024-03-24T12:12:29.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-03-24T12:12:29.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (1290 ms) 2024-03-24T12:12:29.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-03-24T12:12:29.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (10 ms) 2024-03-24T12:12:29.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-03-24T12:12:29.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1019 ms) 2024-03-24T12:12:29.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-03-24T12:12:29.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1073 ms) 2024-03-24T12:12:29.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-03-24T12:12:29.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (4 ms) 2024-03-24T12:12:29.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-03-24T12:12:29.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (3 ms) 2024-03-24T12:12:29.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-03-24T12:12:29.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (4 ms) 2024-03-24T12:12:29.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-03-24T12:12:29.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (3 ms) 2024-03-24T12:12:29.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (3406 ms total) 2024-03-24T12:12:29.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: 2024-03-24T12:12:29.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-03-24T12:12:29.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-03-24T12:12:29.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (1203 ms) 2024-03-24T12:12:29.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-03-24T12:12:29.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (3 ms) 2024-03-24T12:12:29.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-03-24T12:12:29.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1005 ms) 2024-03-24T12:12:29.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-03-24T12:12:29.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1007 ms) 2024-03-24T12:12:29.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-03-24T12:12:29.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (3 ms) 2024-03-24T12:12:29.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-03-24T12:12:29.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (4 ms) 2024-03-24T12:12:29.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-03-24T12:12:29.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (6 ms) 2024-03-24T12:12:29.944 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-03-24T12:12:29.944 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (3 ms) 2024-03-24T12:12:29.944 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (3234 ms total) 2024-03-24T12:12:29.944 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: 2024-03-24T12:12:29.944 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-03-24T12:12:29.944 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (28518 ms total) 2024-03-24T12:12:29.944 INFO:tasks.workunit.client.0.smithi006.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-03-24T12:12:30.776 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: Running main() from gmock_main.cc 2024-03-24T12:12:30.776 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-03-24T12:12:30.776 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-03-24T12:12:30.776 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-03-24T12:12:30.776 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-03-24T12:12:30.777 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: watch_notify_test_cb 2024-03-24T12:12:30.777 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (352 ms) 2024-03-24T12:12:30.777 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-03-24T12:12:30.777 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93905094248368 err -107 2024-03-24T12:12:30.777 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-03-24T12:12:30.777 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (8 ms) 2024-03-24T12:12:30.777 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-03-24T12:12:30.777 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-03-24T12:12:30.777 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93905094963312 err -107 2024-03-24T12:12:30.777 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (1016 ms) 2024-03-24T12:12:30.777 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-03-24T12:12:30.777 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: watch_notify2_test_cb from 24670 notify_id 339302416385 cookie 93905094979840 2024-03-24T12:12:30.777 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (14 ms) 2024-03-24T12:12:30.777 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-03-24T12:12:30.778 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: watch_notify2_test_cb from 24670 notify_id 339302416384 cookie 93905095016304 2024-03-24T12:12:30.778 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (8 ms) 2024-03-24T12:12:30.778 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-03-24T12:12:30.778 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: watch_notify2_test_cb from 24670 notify_id 339302416384 cookie 93905095016304 2024-03-24T12:12:30.778 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (6 ms) 2024-03-24T12:12:30.778 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-03-24T12:12:30.778 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: watch_notify2_test_cb from 24670 notify_id 339302416385 cookie 93905094995792 2024-03-24T12:12:30.778 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: watch_notify2_test_cb from 24670 notify_id 339302416385 cookie 93905095016304 2024-03-24T12:12:30.778 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (7 ms) 2024-03-24T12:12:30.778 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-03-24T12:12:30.778 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: watch_notify2_test_cb from 24670 notify_id 339302416385 cookie 93905095016304 2024-03-24T12:12:30.778 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: watch_notify2_test_cb from 24670 notify_id 343597383682 cookie 93905095016304 2024-03-24T12:12:30.778 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3004 ms) 2024-03-24T12:12:30.778 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-03-24T12:12:30.778 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-03-24T12:12:30.779 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93905094995792 err -107 2024-03-24T12:12:30.779 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: watch_notify2_test_cb from 24670 notify_id 365072220161 cookie 93905094995792 2024-03-24T12:12:30.779 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (4015 ms) 2024-03-24T12:12:30.779 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-03-24T12:12:30.779 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-03-24T12:12:30.779 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93905094995792 err -107 2024-03-24T12:12:30.779 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1005 ms) 2024-03-24T12:12:30.779 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (9435 ms total) 2024-03-24T12:12:30.779 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: 2024-03-24T12:12:30.779 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-03-24T12:12:30.779 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-03-24T12:12:30.779 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: watch_notify_test_cb 2024-03-24T12:12:30.779 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (2177 ms) 2024-03-24T12:12:30.779 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (2177 ms total) 2024-03-24T12:12:30.780 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: 2024-03-24T12:12:30.780 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-03-24T12:12:30.780 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (29262 ms total) 2024-03-24T12:12:30.780 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-03-24T12:12:30.780 INFO:tasks.workunit.client.0.smithi006.stdout: pool: Running main() from gmock_main.cc 2024-03-24T12:12:30.780 INFO:tasks.workunit.client.0.smithi006.stdout: pool: [==========] Running 6 tests from 1 test suite. 2024-03-24T12:12:30.780 INFO:tasks.workunit.client.0.smithi006.stdout: pool: [----------] Global test environment set-up. 2024-03-24T12:12:30.780 INFO:tasks.workunit.client.0.smithi006.stdout: pool: [----------] 6 tests from NeoRadosPools 2024-03-24T12:12:30.781 INFO:tasks.workunit.client.0.smithi006.stdout: pool: [ RUN ] NeoRadosPools.PoolList 2024-03-24T12:12:30.781 INFO:tasks.workunit.client.0.smithi006.stdout: pool: [ OK ] NeoRadosPools.PoolList (10665 ms) 2024-03-24T12:12:30.781 INFO:tasks.workunit.client.0.smithi006.stdout: pool: [ RUN ] NeoRadosPools.PoolLookup 2024-03-24T12:12:30.781 INFO:tasks.workunit.client.0.smithi006.stdout: pool: [ OK ] NeoRadosPools.PoolLookup (5747 ms) 2024-03-24T12:12:30.781 INFO:tasks.workunit.client.0.smithi006.stdout: pool: [ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-03-24T12:12:30.781 INFO:tasks.workunit.client.0.smithi006.stdout: pool: [ OK ] NeoRadosPools.PoolLookupOtherInstance (5081 ms) 2024-03-24T12:12:30.781 INFO:tasks.workunit.client.0.smithi006.stdout: pool: [ RUN ] NeoRadosPools.PoolDelete 2024-03-24T12:12:30.781 INFO:tasks.workunit.client.0.smithi006.stdout: pool: [ OK ] NeoRadosPools.PoolDelete (3542 ms) 2024-03-24T12:12:30.781 INFO:tasks.workunit.client.0.smithi006.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateDelete 2024-03-24T12:12:30.781 INFO:tasks.workunit.client.0.smithi006.stdout: pool: [ OK ] NeoRadosPools.PoolCreateDelete (2074 ms) 2024-03-24T12:12:30.781 INFO:tasks.workunit.client.0.smithi006.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-03-24T12:12:30.781 INFO:tasks.workunit.client.0.smithi006.stdout: pool: [ OK ] NeoRadosPools.PoolCreateWithCrushRule (1874 ms) 2024-03-24T12:12:30.781 INFO:tasks.workunit.client.0.smithi006.stdout: pool: [----------] 6 tests from NeoRadosPools (28984 ms total) 2024-03-24T12:12:30.781 INFO:tasks.workunit.client.0.smithi006.stdout: pool: 2024-03-24T12:12:30.781 INFO:tasks.workunit.client.0.smithi006.stdout: pool: [----------] Global test environment tear-down 2024-03-24T12:12:30.782 INFO:tasks.workunit.client.0.smithi006.stdout: pool: [==========] 6 tests from 1 test suite ran. (28985 ms total) 2024-03-24T12:12:30.782 INFO:tasks.workunit.client.0.smithi006.stdout: pool: [ PASSED ] 6 tests. 2024-03-24T12:12:30.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:30 smithi090 ceph-mon[33646]: pgmap v99: 808 pgs: 70 creating+peering, 190 unknown, 548 active+clean; 145 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 6.2 KiB/s wr, 4 op/s 2024-03-24T12:12:30.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:30 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:30.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:30 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi006-82972-2"}]': finished 2024-03-24T12:12:30.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:30 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "overlaypool": "test-rados-api-smithi006-82271-6"}]': finished 2024-03-24T12:12:30.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:30 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi006-81895-10"}]': finished 2024-03-24T12:12:30.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:30 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]': finished 2024-03-24T12:12:30.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:30 smithi090 ceph-mon[33646]: osdmap e95: 8 total, 8 up, 8 in 2024-03-24T12:12:30.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:30 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82271-6", "mode": "writeback"}]: dispatch 2024-03-24T12:12:30.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:30 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/370521642' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:30.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:30 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82271-6", "mode": "writeback"}]: dispatch 2024-03-24T12:12:30.931 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:30 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:30.931 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:30 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/109588017' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi006-82972-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:30.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[30108]: pgmap v99: 808 pgs: 70 creating+peering, 190 unknown, 548 active+clean; 145 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 6.2 KiB/s wr, 4 op/s 2024-03-24T12:12:30.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:30.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi006-82972-2"}]': finished 2024-03-24T12:12:30.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "overlaypool": "test-rados-api-smithi006-82271-6"}]': finished 2024-03-24T12:12:30.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi006-81895-10"}]': finished 2024-03-24T12:12:30.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]': finished 2024-03-24T12:12:30.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[30108]: osdmap e95: 8 total, 8 up, 8 in 2024-03-24T12:12:30.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82271-6", "mode": "writeback"}]: dispatch 2024-03-24T12:12:30.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/370521642' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:30.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82271-6", "mode": "writeback"}]: dispatch 2024-03-24T12:12:30.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:30.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/109588017' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi006-82972-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:30.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[41734]: pgmap v99: 808 pgs: 70 creating+peering, 190 unknown, 548 active+clean; 145 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 6.2 KiB/s wr, 4 op/s 2024-03-24T12:12:30.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:30.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi006-82972-2"}]': finished 2024-03-24T12:12:30.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "overlaypool": "test-rados-api-smithi006-82271-6"}]': finished 2024-03-24T12:12:30.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi006-81895-10"}]': finished 2024-03-24T12:12:30.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi006-82180-12"}]': finished 2024-03-24T12:12:30.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[41734]: osdmap e95: 8 total, 8 up, 8 in 2024-03-24T12:12:30.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82271-6", "mode": "writeback"}]: dispatch 2024-03-24T12:12:30.965 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/370521642' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:30.965 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82271-6", "mode": "writeback"}]: dispatch 2024-03-24T12:12:30.965 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi006-82180-12"}]: dispatch 2024-03-24T12:12:30.965 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:30 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/109588017' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi006-82972-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:32.091 INFO:tasks.workunit.client.0.smithi006.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-03-24T12:12:32.091 INFO:tasks.workunit.client.0.smithi006.stdout: api_service_pp: [----------] Global test environment set-up. 2024-03-24T12:12:32.091 INFO:tasks.workunit.client.0.smithi006.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-03-24T12:12:32.091 INFO:tasks.workunit.client.0.smithi006.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-03-24T12:12:32.091 INFO:tasks.workunit.client.0.smithi006.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5375 ms) 2024-03-24T12:12:32.091 INFO:tasks.workunit.client.0.smithi006.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-03-24T12:12:32.091 INFO:tasks.workunit.client.0.smithi006.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (25 ms) 2024-03-24T12:12:32.091 INFO:tasks.workunit.client.0.smithi006.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-03-24T12:12:32.091 INFO:tasks.workunit.client.0.smithi006.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20017 ms) 2024-03-24T12:12:32.091 INFO:tasks.workunit.client.0.smithi006.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-03-24T12:12:32.092 INFO:tasks.workunit.client.0.smithi006.stdout: api_service_pp: attempt 0 of 20 2024-03-24T12:12:32.092 INFO:tasks.workunit.client.0.smithi006.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (5046 ms) 2024-03-24T12:12:32.092 INFO:tasks.workunit.client.0.smithi006.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (30463 ms total) 2024-03-24T12:12:32.092 INFO:tasks.workunit.client.0.smithi006.stdout: api_service_pp: 2024-03-24T12:12:32.092 INFO:tasks.workunit.client.0.smithi006.stdout: api_service_pp: [----------] Global test environment tear-down 2024-03-24T12:12:32.092 INFO:tasks.workunit.client.0.smithi006.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (30464 ms total) 2024-03-24T12:12:32.092 INFO:tasks.workunit.client.0.smithi006.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-03-24T12:12:32.144 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:31 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi006-82972-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:32.144 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:32.144 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:31 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:12:32.144 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:31 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82271-6", "mode": "writeback"}]': finished 2024-03-24T12:12:32.144 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:31 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi006-82180-12"}]': finished 2024-03-24T12:12:32.144 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:31 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi006-82972-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:32.144 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/109588017' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi006-82972-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi006-82972-3"}]: dispatch 2024-03-24T12:12:32.145 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4022423322' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:32.145 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/650437844' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:32.145 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:31 smithi090 ceph-mon[33646]: osdmap e96: 8 total, 8 up, 8 in 2024-03-24T12:12:32.145 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:31 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:32.145 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:31 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi006-82972-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:12:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82271-6", "mode": "writeback"}]': finished 2024-03-24T12:12:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi006-82180-12"}]': finished 2024-03-24T12:12:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi006-82972-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/109588017' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi006-82972-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi006-82972-3"}]: dispatch 2024-03-24T12:12:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4022423322' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/650437844' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[30108]: osdmap e96: 8 total, 8 up, 8 in 2024-03-24T12:12:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:32.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi006-82972-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:32.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:32.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:12:32.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82271-6", "mode": "writeback"}]': finished 2024-03-24T12:12:32.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi006-82180-12"}]': finished 2024-03-24T12:12:32.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi006-82972-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:32.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/109588017' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi006-82972-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi006-82972-3"}]: dispatch 2024-03-24T12:12:32.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4022423322' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:32.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/650437844' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:32.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[41734]: osdmap e96: 8 total, 8 up, 8 in 2024-03-24T12:12:32.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:32.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:31 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:32.429 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:12:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:12:33.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:32 smithi090 ceph-mon[33646]: pgmap v102: 616 pgs: 132 unknown, 484 active+clean; 145 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-24T12:12:33.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:32 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:33.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:32 smithi090 ceph-mon[33646]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:33.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:32 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi006-82972-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi006-82972-3"}]: dispatch 2024-03-24T12:12:33.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:32 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:33.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:32 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:33.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:32 smithi090 ceph-mon[33646]: osdmap e97: 8 total, 8 up, 8 in 2024-03-24T12:12:33.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:32 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3879056670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi006-81775-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:33.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:32 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi006-81775-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:33.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:32 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3097152209' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-24T12:12:33.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:32 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-24T12:12:33.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:32 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:12:33.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:32 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:33.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:32 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:12:33.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:32 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:33.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[41734]: pgmap v102: 616 pgs: 132 unknown, 484 active+clean; 145 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-24T12:12:33.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:33.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[41734]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:33.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi006-82972-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi006-82972-3"}]: dispatch 2024-03-24T12:12:33.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:33.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:33.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[41734]: osdmap e97: 8 total, 8 up, 8 in 2024-03-24T12:12:33.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3879056670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi006-81775-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:33.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi006-81775-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:33.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3097152209' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-24T12:12:33.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-24T12:12:33.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:12:33.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:33.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:12:33.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:33.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[30108]: pgmap v102: 616 pgs: 132 unknown, 484 active+clean; 145 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-24T12:12:33.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:33.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[30108]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:33.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi006-82972-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi006-82972-3"}]: dispatch 2024-03-24T12:12:33.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:33.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:33.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[30108]: osdmap e97: 8 total, 8 up, 8 in 2024-03-24T12:12:33.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3879056670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi006-81775-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:33.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi006-81775-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:33.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3097152209' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-24T12:12:33.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-24T12:12:33.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:12:33.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:33.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:12:33.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:32 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:33.268 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: [==========] Running 4 tests from 1 test suite. 2024-03-24T12:12:33.268 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: [----------] Global test environment set-up. 2024-03-24T12:12:33.269 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: [----------] 4 tests from LibRadosService 2024-03-24T12:12:33.269 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-03-24T12:12:33.269 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5049 ms) 2024-03-24T12:12:33.269 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-03-24T12:12:33.269 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: [ OK ] LibRadosService.RegisterLate (29 ms) 2024-03-24T12:12:33.269 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-03-24T12:12:33.269 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: cluster: 2024-03-24T12:12:33.269 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: id: 66d0ad74-e9d6-11ee-95cd-87774f69a715 2024-03-24T12:12:33.269 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: health: HEALTH_WARN 2024-03-24T12:12:33.269 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: 12 pool(s) do not have an application enabled 2024-03-24T12:12:33.269 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: 2024-03-24T12:12:33.269 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: services: 2024-03-24T12:12:33.269 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-03-24T12:12:33.269 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-03-24T12:12:33.270 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-03-24T12:12:33.270 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: foo: 8 portals active (1 hosts, 3 zones) 2024-03-24T12:12:33.270 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-03-24T12:12:33.270 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-03-24T12:12:33.270 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: 2024-03-24T12:12:33.270 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: data: 2024-03-24T12:12:33.270 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: pools: 18 pools, 516 pgs 2024-03-24T12:12:33.270 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: objects: 198 objects, 582 KiB 2024-03-24T12:12:33.270 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: usage: 240 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:12:33.270 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: pgs: 13.953% pgs unknown 2024-03-24T12:12:33.270 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: 48.450% pgs not active 2024-03-24T12:12:33.270 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: 250 creating+peering 2024-03-24T12:12:33.270 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: 194 active+clean 2024-03-24T12:12:33.270 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: 72 unknown 2024-03-24T12:12:33.271 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: 2024-03-24T12:12:33.271 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: io: 2024-03-24T12:12:33.271 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: client: 785 B/s rd, 0 op/s rd, 0 op/s wr 2024-03-24T12:12:33.271 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: 2024-03-24T12:12:33.271 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: 2024-03-24T12:12:33.271 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: cluster: 2024-03-24T12:12:33.271 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: id: 66d0ad74-e9d6-11ee-95cd-87774f69a715 2024-03-24T12:12:33.271 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: health: HEALTH_WARN 2024-03-24T12:12:33.271 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: 26 pool(s) do not have an application enabled 2024-03-24T12:12:33.271 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: 2024-03-24T12:12:33.271 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: services: 2024-03-24T12:12:33.271 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-03-24T12:12:33.271 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-03-24T12:12:33.271 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-03-24T12:12:33.271 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-03-24T12:12:33.272 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-03-24T12:12:33.272 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-03-24T12:12:33.272 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: 2024-03-24T12:12:33.272 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: data: 2024-03-24T12:12:33.272 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: pools: 42 pools, 1284 pgs 2024-03-24T12:12:33.272 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: objects: 198 objects, 582 KiB 2024-03-24T12:12:33.272 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: usage: 240 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:12:33.272 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: pgs: 65.421% pgs unknown 2024-03-24T12:12:33.272 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: 19.470% pgs not active 2024-03-24T12:12:33.272 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: 840 unknown 2024-03-24T12:12:33.272 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: 250 creating+peering 2024-03-24T12:12:33.272 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: 194 active+clean 2024-03-24T12:12:33.272 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: 2024-03-24T12:12:33.272 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: io: 2024-03-24T12:12:33.272 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: client: 794 B/s rd, 0 op/s rd, 0 op/s wr 2024-03-24T12:12:33.273 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: 2024-03-24T12:12:33.273 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: 2024-03-24T12:12:33.273 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: [ OK ] LibRadosService.StatusFormat (6564 ms) 2024-03-24T12:12:33.273 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: [ RUN ] LibRadosService.Status 2024-03-24T12:12:33.273 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: [ OK ] LibRadosService.Status (20035 ms) 2024-03-24T12:12:33.273 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: [----------] 4 tests from LibRadosService (31678 ms total) 2024-03-24T12:12:33.273 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: 2024-03-24T12:12:33.273 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: [----------] Global test environment tear-down 2024-03-24T12:12:33.273 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: [==========] 4 tests from 1 test suite ran. (31678 ms total) 2024-03-24T12:12:33.273 INFO:tasks.workunit.client.0.smithi006.stdout: api_service: [ PASSED ] 4 tests. 2024-03-24T12:12:33.861 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-03-24T12:12:33.861 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-03-24T12:12:33.861 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-03-24T12:12:33.861 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-03-24T12:12:33.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-03-24T12:12:33.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: notify 2024-03-24T12:12:33.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (3305 ms) 2024-03-24T12:12:33.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-03-24T12:12:33.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (1036 ms) 2024-03-24T12:12:33.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (4341 ms total) 2024-03-24T12:12:33.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: 2024-03-24T12:12:33.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-03-24T12:12:33.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-03-24T12:12:33.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: notify 2024-03-24T12:12:33.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (1143 ms) 2024-03-24T12:12:33.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-03-24T12:12:33.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: notify 2024-03-24T12:12:33.863 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (3918 ms) 2024-03-24T12:12:33.863 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-03-24T12:12:33.863 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (5 ms) 2024-03-24T12:12:33.863 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-03-24T12:12:33.863 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (4 ms) 2024-03-24T12:12:33.863 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-03-24T12:12:33.863 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: handle_notify cookie 93916703705776 notify_id 412316860419 notifier_gid 25259 2024-03-24T12:12:33.863 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (5 ms) 2024-03-24T12:12:33.863 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-03-24T12:12:33.863 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: handle_notify cookie 93916703712064 notify_id 412316860420 notifier_gid 25259 2024-03-24T12:12:33.863 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (4 ms) 2024-03-24T12:12:33.863 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-03-24T12:12:33.863 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: handle_notify cookie 93916703705776 notify_id 412316860416 notifier_gid 25259 2024-03-24T12:12:33.863 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (6 ms) 2024-03-24T12:12:33.864 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-03-24T12:12:33.864 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: handle_notify cookie 93916703721776 notify_id 412316860421 notifier_gid 25259 2024-03-24T12:12:33.864 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (4 ms) 2024-03-24T12:12:33.864 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-03-24T12:12:33.864 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: handle_notify cookie 93916703721776 notify_id 412316860422 notifier_gid 25259 2024-03-24T12:12:33.864 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (4 ms) 2024-03-24T12:12:33.864 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-03-24T12:12:33.864 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: handle_notify cookie 93916703721776 notify_id 412316860417 notifier_gid 25259 2024-03-24T12:12:33.864 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (5 ms) 2024-03-24T12:12:33.864 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-03-24T12:12:33.864 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: trying... 2024-03-24T12:12:33.864 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: handle_notify cookie 93916703721776 notify_id 412316860423 notifier_gid 25259 2024-03-24T12:12:33.864 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: timed out 2024-03-24T12:12:33.865 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: flushing 2024-03-24T12:12:34.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:33 smithi090 ceph-mon[33646]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-24T12:12:34.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:33 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi006-81775-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:34.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:33 smithi090 ceph-mon[33646]: osdmap e98: 8 total, 8 up, 8 in 2024-03-24T12:12:34.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:33 smithi006 ceph-mon[30108]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-24T12:12:34.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:33 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi006-81775-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:34.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:33 smithi006 ceph-mon[30108]: osdmap e98: 8 total, 8 up, 8 in 2024-03-24T12:12:34.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:33 smithi006 ceph-mon[41734]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-24T12:12:34.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:33 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi006-81775-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:34.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:33 smithi006 ceph-mon[41734]: osdmap e98: 8 total, 8 up, 8 in 2024-03-24T12:12:35.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:34 smithi006 ceph-mon[30108]: pgmap v105: 584 pgs: 140 creating+peering, 444 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-24T12:12:35.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:35.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:34 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi006-82972-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi006-82972-3"}]': finished 2024-03-24T12:12:35.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:34 smithi006 ceph-mon[30108]: osdmap e99: 8 total, 8 up, 8 in 2024-03-24T12:12:35.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4261555526' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:35.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:34 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:35.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:35.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:34 smithi006 ceph-mon[41734]: pgmap v105: 584 pgs: 140 creating+peering, 444 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-24T12:12:35.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:35.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:34 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi006-82972-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi006-82972-3"}]': finished 2024-03-24T12:12:35.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:34 smithi006 ceph-mon[41734]: osdmap e99: 8 total, 8 up, 8 in 2024-03-24T12:12:35.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4261555526' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:35.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:34 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:35.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:35.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:34 smithi090 ceph-mon[33646]: pgmap v105: 584 pgs: 140 creating+peering, 444 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-24T12:12:35.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:35.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:34 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi006-82972-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi006-82972-3"}]': finished 2024-03-24T12:12:35.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:34 smithi090 ceph-mon[33646]: osdmap e99: 8 total, 8 up, 8 in 2024-03-24T12:12:35.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4261555526' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:35.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:34 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:35.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:35.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:12:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:12:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:12:36.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:36 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:36.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:36 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1688204294' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi006-81794-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:36.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:36 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:36.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:36 smithi090 ceph-mon[33646]: pgmap v108: 624 pgs: 200 unknown, 32 creating+peering, 392 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T12:12:36.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:36 smithi090 ceph-mon[33646]: osdmap e100: 8 total, 8 up, 8 in 2024-03-24T12:12:36.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:36 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi006-81794-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:36.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:36 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/41128033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi006-81775-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:36.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:36 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:36.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:36 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:36.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:36 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi006-81794-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:36.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:36 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/41128033' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi006-81775-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:36.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:36 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:36.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:36 smithi090 ceph-mon[33646]: osdmap e101: 8 total, 8 up, 8 in 2024-03-24T12:12:36.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:36.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1688204294' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi006-81794-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:36.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:36.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[30108]: pgmap v108: 624 pgs: 200 unknown, 32 creating+peering, 392 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T12:12:36.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[30108]: osdmap e100: 8 total, 8 up, 8 in 2024-03-24T12:12:36.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi006-81794-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:36.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/41128033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi006-81775-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:36.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:36.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:36.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi006-81794-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:36.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/41128033' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi006-81775-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:36.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:36.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[30108]: osdmap e101: 8 total, 8 up, 8 in 2024-03-24T12:12:36.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:36.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1688204294' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi006-81794-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:36.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:36.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[41734]: pgmap v108: 624 pgs: 200 unknown, 32 creating+peering, 392 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T12:12:36.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[41734]: osdmap e100: 8 total, 8 up, 8 in 2024-03-24T12:12:36.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi006-81794-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:36.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/41128033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi006-81775-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:36.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:36.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:36.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi006-81794-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:36.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/41128033' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi006-81775-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:36.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:36.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:36 smithi006 ceph-mon[41734]: osdmap e101: 8 total, 8 up, 8 in 2024-03-24T12:12:37.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3151810967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:37.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:37 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:37.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:37.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:37 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:37.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:37 smithi006 ceph-mon[30108]: osdmap e102: 8 total, 8 up, 8 in 2024-03-24T12:12:37.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1265083323' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:37.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/109588017' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi006-82972-3"}]: dispatch 2024-03-24T12:12:37.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:37 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:37.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:37.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3151810967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:37.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:37 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:37.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:37.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:37 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:37.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:37 smithi006 ceph-mon[41734]: osdmap e102: 8 total, 8 up, 8 in 2024-03-24T12:12:37.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1265083323' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:37.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/109588017' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi006-82972-3"}]: dispatch 2024-03-24T12:12:37.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:37 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:37.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3151810967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:37 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:37 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:37 smithi090 ceph-mon[33646]: osdmap e102: 8 total, 8 up, 8 in 2024-03-24T12:12:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1265083323' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/109588017' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi006-82972-3"}]: dispatch 2024-03-24T12:12:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:37 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:38.961 INFO:tasks.workunit.client.0.smithi006.stdout: api_ ec_list: Running main() from gmock_main.cc 2024-03-24T12:12:38.961 INFO:tasks.workunit.client.0.smithi006.stdout: ec_list: [==========] Running 3 tests from 1 test suite. 2024-03-24T12:12:38.961 INFO:tasks.workunit.client.0.smithi006.stdout: ec_list: [----------] Global test environment set-up. 2024-03-24T12:12:38.961 INFO:tasks.workunit.client.0.smithi006.stdout: ec_list: [----------] 3 tests from NeoradosECList 2024-03-24T12:12:38.961 INFO:tasks.workunit.client.0.smithi006.stdout: ec_list: [ RUN ] NeoradosECList.ListObjects 2024-03-24T12:12:38.962 INFO:tasks.workunit.client.0.smithi006.stdout: ec_list: [ OK ] NeoradosECList.ListObjects (19846 ms) 2024-03-24T12:12:38.962 INFO:tasks.workunit.client.0.smithi006.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsNS 2024-03-24T12:12:38.962 INFO:tasks.workunit.client.0.smithi006.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsNS (8291 ms) 2024-03-24T12:12:38.962 INFO:tasks.workunit.client.0.smithi006.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsMany 2024-03-24T12:12:38.962 INFO:tasks.workunit.client.0.smithi006.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsMany (9043 ms) 2024-03-24T12:12:38.962 INFO:tasks.workunit.client.0.smithi006.stdout: ec_list: [----------] 3 tests from NeoradosECList (37180 ms total) 2024-03-24T12:12:38.962 INFO:tasks.workunit.client.0.smithi006.stdout: ec_list: 2024-03-24T12:12:38.962 INFO:tasks.workunit.client.0.smithi006.stdout: ec_list: [----------] Global test environment tear-down 2024-03-24T12:12:38.962 INFO:tasks.workunit.client.0.smithi006.stdout: ec_list: [==========] 3 tests from 1 test suite ran. (37180 ms total) 2024-03-24T12:12:38.962 INFO:tasks.workunit.client.0.smithi006.stdout: ec_list: [ PASSED ] 3 tests. 2024-03-24T12:12:38.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[30108]: pgmap v111: 616 pgs: 39 creating+peering, 187 unknown, 390 active+clean; 145 MiB data, 760 MiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-03-24T12:12:38.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:38.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi006-82972-3"}]: dispatch 2024-03-24T12:12:38.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:38.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:38.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi006-82972-3"}]': finished 2024-03-24T12:12:38.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:38.964 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[30108]: osdmap e103: 8 total, 8 up, 8 in 2024-03-24T12:12:38.964 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/420251482' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi006-82137-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:38.964 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi006-82137-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:38.964 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/109588017' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi006-82972-3"}]: dispatch 2024-03-24T12:12:38.964 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3320351002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi006-81775-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:38.964 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1960546628' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi006-81794-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:38.964 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:38.964 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi006-81794-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:38.964 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi006-82972-3"}]: dispatch 2024-03-24T12:12:38.964 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi006-81775-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:38.964 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:38.964 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:38.965 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[41734]: pgmap v111: 616 pgs: 39 creating+peering, 187 unknown, 390 active+clean; 145 MiB data, 760 MiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-03-24T12:12:38.965 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:38.965 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi006-82972-3"}]: dispatch 2024-03-24T12:12:38.965 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:38.965 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:38.965 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi006-82972-3"}]': finished 2024-03-24T12:12:38.965 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:38.965 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[41734]: osdmap e103: 8 total, 8 up, 8 in 2024-03-24T12:12:38.965 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/420251482' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi006-82137-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:38.965 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi006-82137-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:38.965 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/109588017' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi006-82972-3"}]: dispatch 2024-03-24T12:12:38.966 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3320351002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi006-81775-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:38.966 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1960546628' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi006-81794-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:38.966 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:38.966 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi006-81794-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:38.966 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi006-82972-3"}]: dispatch 2024-03-24T12:12:38.966 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi006-81775-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:38.966 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:38.966 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:38 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:39.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:38 smithi090 ceph-mon[33646]: pgmap v111: 616 pgs: 39 creating+peering, 187 unknown, 390 active+clean; 145 MiB data, 760 MiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-03-24T12:12:39.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:38 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:39.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi006-82972-3"}]: dispatch 2024-03-24T12:12:39.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:39.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82442-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:39.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi006-82972-3"}]': finished 2024-03-24T12:12:39.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:39.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:38 smithi090 ceph-mon[33646]: osdmap e103: 8 total, 8 up, 8 in 2024-03-24T12:12:39.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:38 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/420251482' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi006-82137-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:39.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi006-82137-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:39.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:38 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/109588017' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi006-82972-3"}]: dispatch 2024-03-24T12:12:39.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:38 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3320351002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi006-81775-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:39.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:38 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1960546628' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi006-81794-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:39.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:38 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:39.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi006-81794-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:39.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi006-82972-3"}]: dispatch 2024-03-24T12:12:39.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi006-81775-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:39.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:39.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:38 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:39.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:39 smithi006 ceph-mon[30108]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:39.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi006-82137-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:39.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi006-81794-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:39.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi006-82972-3"}]': finished 2024-03-24T12:12:39.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi006-81775-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:39.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-7", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:12:39.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:39 smithi006 ceph-mon[30108]: osdmap e104: 8 total, 8 up, 8 in 2024-03-24T12:12:39.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-7"}]: dispatch 2024-03-24T12:12:39.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-7"}]: dispatch 2024-03-24T12:12:39.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:39 smithi006 ceph-mon[30108]: pgmap v114: 680 pgs: 96 creating+peering, 160 unknown, 424 active+clean; 145 MiB data, 748 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:12:39.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:39.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:39 smithi006 ceph-mon[41734]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:39.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi006-82137-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:39.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi006-81794-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:39.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi006-82972-3"}]': finished 2024-03-24T12:12:39.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi006-81775-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:39.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-7", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:12:39.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:39 smithi006 ceph-mon[41734]: osdmap e104: 8 total, 8 up, 8 in 2024-03-24T12:12:39.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:39 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-7"}]: dispatch 2024-03-24T12:12:39.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-7"}]: dispatch 2024-03-24T12:12:39.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:39 smithi006 ceph-mon[41734]: pgmap v114: 680 pgs: 96 creating+peering, 160 unknown, 424 active+clean; 145 MiB data, 748 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:12:39.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:39 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:39.983 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-03-24T12:12:39.983 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-03-24T12:12:39.983 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-03-24T12:12:39.984 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-03-24T12:12:39.984 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-03-24T12:12:39.984 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-03-24T12:12:39.984 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-03-24T12:12:39.984 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: 2024-03-24T12:12:39.984 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-03-24T12:12:39.984 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-03-24T12:12:39.984 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (11560 ms) 2024-03-24T12:12:39.984 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-03-24T12:12:39.984 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (8487 ms) 2024-03-24T12:12:39.984 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-03-24T12:12:39.984 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (5199 ms) 2024-03-24T12:12:39.984 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-03-24T12:12:39.984 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (3071 ms) 2024-03-24T12:12:39.985 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-03-24T12:12:39.985 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (3194 ms) 2024-03-24T12:12:39.985 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-03-24T12:12:39.985 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (2705 ms) 2024-03-24T12:12:39.985 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-03-24T12:12:39.985 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (4176 ms) 2024-03-24T12:12:39.985 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (38392 ms total) 2024-03-24T12:12:39.985 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: 2024-03-24T12:12:39.985 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-03-24T12:12:39.985 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (38392 ms total) 2024-03-24T12:12:39.985 INFO:tasks.workunit.client.0.smithi006.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-03-24T12:12:40.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:39 smithi090 ceph-mon[33646]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:40.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi006-82137-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:40.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi006-81794-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:40.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi006-82972-3"}]': finished 2024-03-24T12:12:40.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi006-81775-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:40.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-7", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:12:40.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:39 smithi090 ceph-mon[33646]: osdmap e104: 8 total, 8 up, 8 in 2024-03-24T12:12:40.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:39 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-7"}]: dispatch 2024-03-24T12:12:40.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-7"}]: dispatch 2024-03-24T12:12:40.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:39 smithi090 ceph-mon[33646]: pgmap v114: 680 pgs: 96 creating+peering, 160 unknown, 424 active+clean; 145 MiB data, 748 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:12:40.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:39 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:41.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:41 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-7"}]': finished 2024-03-24T12:12:41.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:41 smithi090 ceph-mon[33646]: osdmap e105: 8 total, 8 up, 8 in 2024-03-24T12:12:41.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:12:41.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:41.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:41 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:12:41.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:41 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:12:41.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:41 smithi090 ceph-mon[33646]: osdmap e106: 8 total, 8 up, 8 in 2024-03-24T12:12:41.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2449792320' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi006-81794-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:41.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-7"}]: dispatch 2024-03-24T12:12:41.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1567217524' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi006-81775-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:41.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:41 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-7"}]: dispatch 2024-03-24T12:12:41.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2709173721' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:41.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:41 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:41.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-7"}]': finished 2024-03-24T12:12:41.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[30108]: osdmap e105: 8 total, 8 up, 8 in 2024-03-24T12:12:41.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:12:41.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:41.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:12:41.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:12:41.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[30108]: osdmap e106: 8 total, 8 up, 8 in 2024-03-24T12:12:41.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2449792320' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi006-81794-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:41.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-7"}]: dispatch 2024-03-24T12:12:41.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1567217524' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi006-81775-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:41.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-7"}]: dispatch 2024-03-24T12:12:41.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2709173721' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:41.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:41.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-7"}]': finished 2024-03-24T12:12:41.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[41734]: osdmap e105: 8 total, 8 up, 8 in 2024-03-24T12:12:41.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:12:41.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:41.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:12:41.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:12:41.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[41734]: osdmap e106: 8 total, 8 up, 8 in 2024-03-24T12:12:41.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2449792320' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi006-81794-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:41.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-7"}]: dispatch 2024-03-24T12:12:41.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1567217524' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi006-81775-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:41.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-7"}]: dispatch 2024-03-24T12:12:41.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2709173721' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:41.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:41 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:42.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:12:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:12:42.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:42 smithi090 ceph-mon[33646]: pgmap v117: 648 pgs: 64 creating+peering, 192 unknown, 392 active+clean; 145 MiB data, 748 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:12:42.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:42 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:42.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:42 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2449792320' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi006-81794-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:42.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:42 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1567217524' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi006-81775-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:42.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:42 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-7"}]': finished 2024-03-24T12:12:42.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:42 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:42.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:42 smithi090 ceph-mon[33646]: osdmap e107: 8 total, 8 up, 8 in 2024-03-24T12:12:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:42 smithi006 ceph-mon[30108]: pgmap v117: 648 pgs: 64 creating+peering, 192 unknown, 392 active+clean; 145 MiB data, 748 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:12:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:42 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:42 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2449792320' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi006-81794-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:42 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1567217524' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi006-81775-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:42 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-7"}]': finished 2024-03-24T12:12:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:42 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:42 smithi006 ceph-mon[30108]: osdmap e107: 8 total, 8 up, 8 in 2024-03-24T12:12:42.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:42 smithi006 ceph-mon[41734]: pgmap v117: 648 pgs: 64 creating+peering, 192 unknown, 392 active+clean; 145 MiB data, 748 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:12:42.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:42 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:42.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:42 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2449792320' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi006-81794-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:42.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:42 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1567217524' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi006-81775-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:42.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:42 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-7"}]': finished 2024-03-24T12:12:42.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:42 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:42.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:42 smithi006 ceph-mon[41734]: osdmap e107: 8 total, 8 up, 8 in 2024-03-24T12:12:42.806 INFO:tasks.workunit.client.0.smithi006.stdout: write_operations: Running main() from gmock_main.cc 2024-03-24T12:12:42.806 INFO:tasks.workunit.client.0.smithi006.stdout: write_operations: [==========] Running 7 tests from 1 test suite. 2024-03-24T12:12:42.806 INFO:tasks.workunit.client.0.smithi006.stdout: write_operations: [----------] Global test environment set-up. 2024-03-24T12:12:42.806 INFO:tasks.workunit.client.0.smithi006.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps 2024-03-24T12:12:42.806 INFO:tasks.workunit.client.0.smithi006.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertExists 2024-03-24T12:12:42.806 INFO:tasks.workunit.client.0.smithi006.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertExists (11348 ms) 2024-03-24T12:12:42.806 INFO:tasks.workunit.client.0.smithi006.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertVersion 2024-03-24T12:12:42.807 INFO:tasks.workunit.client.0.smithi006.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertVersion (8476 ms) 2024-03-24T12:12:42.807 INFO:tasks.workunit.client.0.smithi006.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Xattrs 2024-03-24T12:12:42.807 INFO:tasks.workunit.client.0.smithi006.stdout: write_operations: [ OK ] NeoRadosWriteOps.Xattrs (3592 ms) 2024-03-24T12:12:42.807 INFO:tasks.workunit.client.0.smithi006.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Write 2024-03-24T12:12:42.807 INFO:tasks.workunit.client.0.smithi006.stdout: write_operations: [ OK ] NeoRadosWriteOps.Write (4688 ms) 2024-03-24T12:12:42.807 INFO:tasks.workunit.client.0.smithi006.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Exec 2024-03-24T12:12:42.807 INFO:tasks.workunit.client.0.smithi006.stdout: write_operations: [ OK ] NeoRadosWriteOps.Exec (5052 ms) 2024-03-24T12:12:42.807 INFO:tasks.workunit.client.0.smithi006.stdout: write_operations: [ RUN ] NeoRadosWriteOps.WriteSame 2024-03-24T12:12:42.807 INFO:tasks.workunit.client.0.smithi006.stdout: write_operations: [ OK ] NeoRadosWriteOps.WriteSame (4000 ms) 2024-03-24T12:12:42.807 INFO:tasks.workunit.client.0.smithi006.stdout: write_operations: [ RUN ] NeoRadosWriteOps.CmpExt 2024-03-24T12:12:42.807 INFO:tasks.workunit.client.0.smithi006.stdout: write_operations: [ OK ] NeoRadosWriteOps.CmpExt (3842 ms) 2024-03-24T12:12:42.807 INFO:tasks.workunit.client.0.smithi006.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps (40998 ms total) 2024-03-24T12:12:42.807 INFO:tasks.workunit.client.0.smithi006.stdout: write_operations: 2024-03-24T12:12:42.807 INFO:tasks.workunit.client.0.smithi006.stdout: write_operations: [----------] Global test environment tear-down 2024-03-24T12:12:42.807 INFO:tasks.workunit.client.0.smithi006.stdout: write_operations: [==========] 7 tests from 1 test suite ran. (40998 ms total) 2024-03-24T12:12:42.808 INFO:tasks.workunit.client.0.smithi006.stdout: write_operations: [ PASSED ] 7 tests. 2024-03-24T12:12:44.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:43 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:12:44.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:43 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:44.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:43 smithi090 ceph-mon[33646]: osdmap e108: 8 total, 8 up, 8 in 2024-03-24T12:12:44.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:43 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4"}]: dispatch 2024-03-24T12:12:44.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:43 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4"}]: dispatch 2024-03-24T12:12:44.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:43 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:12:44.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:43 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:44.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:43 smithi006 ceph-mon[30108]: osdmap e108: 8 total, 8 up, 8 in 2024-03-24T12:12:44.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:43 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4"}]: dispatch 2024-03-24T12:12:44.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:43 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4"}]: dispatch 2024-03-24T12:12:44.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:43 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:12:44.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:43 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:44.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:43 smithi006 ceph-mon[41734]: osdmap e108: 8 total, 8 up, 8 in 2024-03-24T12:12:44.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:43 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4"}]: dispatch 2024-03-24T12:12:44.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:43 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4"}]: dispatch 2024-03-24T12:12:45.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:44 smithi090 ceph-mon[33646]: pgmap v120: 584 pgs: 64 creating+peering, 64 unknown, 456 active+clean; 145 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-24T12:12:45.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:44 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:45.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:44 smithi090 ceph-mon[33646]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:45.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:44 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4"}]': finished 2024-03-24T12:12:45.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:44 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "tierpool": "test-rados-api-smithi006-82271-6"}]: dispatch 2024-03-24T12:12:45.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:44 smithi090 ceph-mon[33646]: osdmap e109: 8 total, 8 up, 8 in 2024-03-24T12:12:45.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:44 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/893881563' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi006-81775-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:45.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:44 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "tierpool": "test-rados-api-smithi006-82271-6"}]: dispatch 2024-03-24T12:12:45.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:44 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:45.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:44 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi006-81775-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:45.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:44 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2135835980' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi006-81794-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:45.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:44 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:45.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:44 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:45.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[30108]: pgmap v120: 584 pgs: 64 creating+peering, 64 unknown, 456 active+clean; 145 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-24T12:12:45.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:45.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[30108]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:45.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4"}]': finished 2024-03-24T12:12:45.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "tierpool": "test-rados-api-smithi006-82271-6"}]: dispatch 2024-03-24T12:12:45.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[30108]: osdmap e109: 8 total, 8 up, 8 in 2024-03-24T12:12:45.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/893881563' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi006-81775-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:45.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "tierpool": "test-rados-api-smithi006-82271-6"}]: dispatch 2024-03-24T12:12:45.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:45.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi006-81775-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:45.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2135835980' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi006-81794-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:45.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:45.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:45.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[41734]: pgmap v120: 584 pgs: 64 creating+peering, 64 unknown, 456 active+clean; 145 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-24T12:12:45.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:45.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[41734]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:45.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4"}]': finished 2024-03-24T12:12:45.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "tierpool": "test-rados-api-smithi006-82271-6"}]: dispatch 2024-03-24T12:12:45.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[41734]: osdmap e109: 8 total, 8 up, 8 in 2024-03-24T12:12:45.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/893881563' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi006-81775-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:45.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "tierpool": "test-rados-api-smithi006-82271-6"}]: dispatch 2024-03-24T12:12:45.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:45.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi006-81775-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:45.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2135835980' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi006-81794-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:45.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:45.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:44 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:45.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:12:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:12:46.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:12:46.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "tierpool": "test-rados-api-smithi006-82271-6"}]': finished 2024-03-24T12:12:46.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi006-81775-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:46.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2135835980' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi006-81794-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:46.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:46.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi006-82271-6", "pool2": "test-rados-api-smithi006-82271-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:12:46.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: osdmap e110: 8 total, 8 up, 8 in 2024-03-24T12:12:46.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi006-82271-6", "pool2": "test-rados-api-smithi006-82271-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:12:46.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:46.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.0"}]: dispatch 2024-03-24T12:12:46.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.0"}]: dispatch 2024-03-24T12:12:46.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.1"}]: dispatch 2024-03-24T12:12:46.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.1"}]: dispatch 2024-03-24T12:12:46.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.2"}]: dispatch 2024-03-24T12:12:46.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.2"}]: dispatch 2024-03-24T12:12:46.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.3"}]: dispatch 2024-03-24T12:12:46.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.3"}]: dispatch 2024-03-24T12:12:46.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.4"}]: dispatch 2024-03-24T12:12:46.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.4"}]: dispatch 2024-03-24T12:12:46.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.5"}]: dispatch 2024-03-24T12:12:46.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.5"}]: dispatch 2024-03-24T12:12:46.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.6"}]: dispatch 2024-03-24T12:12:46.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.6"}]: dispatch 2024-03-24T12:12:46.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.7"}]: dispatch 2024-03-24T12:12:46.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.7"}]: dispatch 2024-03-24T12:12:46.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.8"}]: dispatch 2024-03-24T12:12:46.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.8"}]: dispatch 2024-03-24T12:12:46.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.9"}]: dispatch 2024-03-24T12:12:46.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.9"}]: dispatch 2024-03-24T12:12:46.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: pgmap v123: 616 pgs: 192 unknown, 424 active+clean; 145 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-24T12:12:46.181 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: 21.8 deep-scrub starts 2024-03-24T12:12:46.181 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: 21.8 deep-scrub ok 2024-03-24T12:12:46.181 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: 21.7 deep-scrub starts 2024-03-24T12:12:46.181 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:46.181 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: 21.7 deep-scrub ok 2024-03-24T12:12:46.181 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: 21.5 deep-scrub starts 2024-03-24T12:12:46.181 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: 21.5 deep-scrub ok 2024-03-24T12:12:46.181 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: 21.9 deep-scrub starts 2024-03-24T12:12:46.181 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: 21.9 deep-scrub ok 2024-03-24T12:12:46.181 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:46.181 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi006-82271-6", "pool2": "test-rados-api-smithi006-82271-6", "yes_i_really_really_mean_it": true}]': finished 2024-03-24T12:12:46.182 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-9", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:12:46.182 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-9"}]: dispatch 2024-03-24T12:12:46.182 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: osdmap e111: 8 total, 8 up, 8 in 2024-03-24T12:12:46.182 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1869995198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:46.182 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-9"}]: dispatch 2024-03-24T12:12:46.182 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:12:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "tierpool": "test-rados-api-smithi006-82271-6"}]': finished 2024-03-24T12:12:46.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi006-81775-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:46.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2135835980' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi006-81794-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:46.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:46.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi006-82271-6", "pool2": "test-rados-api-smithi006-82271-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:12:46.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: osdmap e110: 8 total, 8 up, 8 in 2024-03-24T12:12:46.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi006-82271-6", "pool2": "test-rados-api-smithi006-82271-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:12:46.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:46.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.0"}]: dispatch 2024-03-24T12:12:46.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.0"}]: dispatch 2024-03-24T12:12:46.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.1"}]: dispatch 2024-03-24T12:12:46.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.1"}]: dispatch 2024-03-24T12:12:46.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.2"}]: dispatch 2024-03-24T12:12:46.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.2"}]: dispatch 2024-03-24T12:12:46.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.3"}]: dispatch 2024-03-24T12:12:46.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.3"}]: dispatch 2024-03-24T12:12:46.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.4"}]: dispatch 2024-03-24T12:12:46.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.4"}]: dispatch 2024-03-24T12:12:46.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.5"}]: dispatch 2024-03-24T12:12:46.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.5"}]: dispatch 2024-03-24T12:12:46.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.6"}]: dispatch 2024-03-24T12:12:46.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.6"}]: dispatch 2024-03-24T12:12:46.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:12:46.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi006-82271-4", "tierpool": "test-rados-api-smithi006-82271-6"}]': finished 2024-03-24T12:12:46.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi006-81775-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:46.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2135835980' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi006-81794-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:46.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:46.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/520662618' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi006-82271-6", "pool2": "test-rados-api-smithi006-82271-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:12:46.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: osdmap e110: 8 total, 8 up, 8 in 2024-03-24T12:12:46.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi006-82271-6", "pool2": "test-rados-api-smithi006-82271-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:12:46.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:46.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.0"}]: dispatch 2024-03-24T12:12:46.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.0"}]: dispatch 2024-03-24T12:12:46.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.1"}]: dispatch 2024-03-24T12:12:46.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.1"}]: dispatch 2024-03-24T12:12:46.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.2"}]: dispatch 2024-03-24T12:12:46.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.2"}]: dispatch 2024-03-24T12:12:46.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.3"}]: dispatch 2024-03-24T12:12:46.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.3"}]: dispatch 2024-03-24T12:12:46.215 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.4"}]: dispatch 2024-03-24T12:12:46.215 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.4"}]: dispatch 2024-03-24T12:12:46.215 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.5"}]: dispatch 2024-03-24T12:12:46.215 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.5"}]: dispatch 2024-03-24T12:12:46.215 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.6"}]: dispatch 2024-03-24T12:12:46.215 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.6"}]: dispatch 2024-03-24T12:12:46.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.7"}]: dispatch 2024-03-24T12:12:46.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.7"}]: dispatch 2024-03-24T12:12:46.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.8"}]: dispatch 2024-03-24T12:12:46.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.8"}]: dispatch 2024-03-24T12:12:46.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.9"}]: dispatch 2024-03-24T12:12:46.216 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.9"}]: dispatch 2024-03-24T12:12:46.216 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: pgmap v123: 616 pgs: 192 unknown, 424 active+clean; 145 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-24T12:12:46.216 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: 21.8 deep-scrub starts 2024-03-24T12:12:46.216 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: 21.8 deep-scrub ok 2024-03-24T12:12:46.216 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: 21.7 deep-scrub starts 2024-03-24T12:12:46.216 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:46.216 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: 21.7 deep-scrub ok 2024-03-24T12:12:46.216 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: 21.5 deep-scrub starts 2024-03-24T12:12:46.216 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: 21.5 deep-scrub ok 2024-03-24T12:12:46.216 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: 21.9 deep-scrub starts 2024-03-24T12:12:46.216 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: 21.9 deep-scrub ok 2024-03-24T12:12:46.217 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:46.217 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi006-82271-6", "pool2": "test-rados-api-smithi006-82271-6", "yes_i_really_really_mean_it": true}]': finished 2024-03-24T12:12:46.217 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-9", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:12:46.217 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-9"}]: dispatch 2024-03-24T12:12:46.217 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: osdmap e111: 8 total, 8 up, 8 in 2024-03-24T12:12:46.217 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1869995198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:46.217 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-9"}]: dispatch 2024-03-24T12:12:46.217 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:46.217 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.7"}]: dispatch 2024-03-24T12:12:46.217 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.7"}]: dispatch 2024-03-24T12:12:46.217 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.8"}]: dispatch 2024-03-24T12:12:46.218 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.8"}]: dispatch 2024-03-24T12:12:46.218 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.9"}]: dispatch 2024-03-24T12:12:46.218 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "21.9"}]: dispatch 2024-03-24T12:12:46.218 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: pgmap v123: 616 pgs: 192 unknown, 424 active+clean; 145 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-24T12:12:46.218 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: 21.8 deep-scrub starts 2024-03-24T12:12:46.218 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: 21.8 deep-scrub ok 2024-03-24T12:12:46.218 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: 21.7 deep-scrub starts 2024-03-24T12:12:46.218 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:46.218 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: 21.7 deep-scrub ok 2024-03-24T12:12:46.218 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: 21.5 deep-scrub starts 2024-03-24T12:12:46.218 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: 21.5 deep-scrub ok 2024-03-24T12:12:46.218 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: 21.9 deep-scrub starts 2024-03-24T12:12:46.218 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: 21.9 deep-scrub ok 2024-03-24T12:12:46.219 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:46.219 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi006-82271-6", "pool2": "test-rados-api-smithi006-82271-6", "yes_i_really_really_mean_it": true}]': finished 2024-03-24T12:12:46.219 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-9", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:12:46.219 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-9"}]: dispatch 2024-03-24T12:12:46.219 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: osdmap e111: 8 total, 8 up, 8 in 2024-03-24T12:12:46.219 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1869995198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:46.219 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-9"}]: dispatch 2024-03-24T12:12:46.219 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:46.819 INFO:tasks.workunit.client.0.smithi006.stdout:watch_notify_pp: flushed 2024-03-24T12:12:46.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3004 ms) 2024-03-24T12:12:46.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-03-24T12:12:46.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: trying... 2024-03-24T12:12:46.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: handle_notify cookie 93916703721776 notify_id 420906795010 notifier_gid 25259 2024-03-24T12:12:46.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: timed out 2024-03-24T12:12:46.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: flushing 2024-03-24T12:12:46.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: flushed 2024-03-24T12:12:46.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3004 ms) 2024-03-24T12:12:46.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-03-24T12:12:46.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: List watches 2024-03-24T12:12:46.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: notify2 2024-03-24T12:12:46.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: handle_notify cookie 93916703721776 notify_id 433791696898 notifier_gid 25259 2024-03-24T12:12:46.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: notify2 done 2024-03-24T12:12:46.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: watch_check 2024-03-24T12:12:46.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: unwatch2 2024-03-24T12:12:46.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: flushing 2024-03-24T12:12:46.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: done 2024-03-24T12:12:46.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3007 ms) 2024-03-24T12:12:46.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-03-24T12:12:46.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: List watches 2024-03-24T12:12:46.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: notify2 2024-03-24T12:12:46.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: handle_notify cookie 93916703725552 notify_id 446676598792 notifier_gid 25259 2024-03-24T12:12:46.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: notify2 done 2024-03-24T12:12:46.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: watch_check 2024-03-24T12:12:46.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: unwatch2 2024-03-24T12:12:46.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: flushing 2024-03-24T12:12:46.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: done 2024-03-24T12:12:46.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3022 ms) 2024-03-24T12:12:46.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (17135 ms total) 2024-03-24T12:12:46.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: 2024-03-24T12:12:46.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-03-24T12:12:46.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (45283 ms total) 2024-03-24T12:12:46.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-03-24T12:12:47.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:47 smithi090 ceph-mon[33646]: 21.4 deep-scrub starts 2024-03-24T12:12:47.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:47 smithi090 ceph-mon[33646]: 21.4 deep-scrub ok 2024-03-24T12:12:47.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:47 smithi090 ceph-mon[33646]: 21.6 deep-scrub starts 2024-03-24T12:12:47.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:47 smithi090 ceph-mon[33646]: 21.6 deep-scrub ok 2024-03-24T12:12:47.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418203406' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:47.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:47 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:47.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418203406' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:47.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:47 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:47.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418203406' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:47.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:47 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:47.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:47.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:47 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-9"}]': finished 2024-03-24T12:12:47.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:47 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:47.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:47 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:47.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-9", "mode": "writeback"}]: dispatch 2024-03-24T12:12:47.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:47 smithi090 ceph-mon[33646]: osdmap e112: 8 total, 8 up, 8 in 2024-03-24T12:12:47.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418203406' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi006-82076-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:47.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1938122044' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi006-81794-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:47.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3261103760' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi006-81775-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:47.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:47 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-9", "mode": "writeback"}]: dispatch 2024-03-24T12:12:47.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[30108]: 21.4 deep-scrub starts 2024-03-24T12:12:47.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[30108]: 21.4 deep-scrub ok 2024-03-24T12:12:47.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[30108]: 21.6 deep-scrub starts 2024-03-24T12:12:47.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[30108]: 21.6 deep-scrub ok 2024-03-24T12:12:47.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418203406' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:47.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:47.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418203406' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:47.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:47.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418203406' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:47.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:47.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:47.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-9"}]': finished 2024-03-24T12:12:47.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:47.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:47.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-9", "mode": "writeback"}]: dispatch 2024-03-24T12:12:47.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[30108]: osdmap e112: 8 total, 8 up, 8 in 2024-03-24T12:12:47.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418203406' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi006-82076-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:47.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1938122044' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi006-81794-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:47.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3261103760' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi006-81775-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:47.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-9", "mode": "writeback"}]: dispatch 2024-03-24T12:12:47.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[41734]: 21.4 deep-scrub starts 2024-03-24T12:12:47.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[41734]: 21.4 deep-scrub ok 2024-03-24T12:12:47.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[41734]: 21.6 deep-scrub starts 2024-03-24T12:12:47.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[41734]: 21.6 deep-scrub ok 2024-03-24T12:12:47.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418203406' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:47.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:47.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418203406' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:47.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:47.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418203406' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:47.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:12:47.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:47.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-9"}]': finished 2024-03-24T12:12:47.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82043-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:47.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:12:47.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-9", "mode": "writeback"}]: dispatch 2024-03-24T12:12:47.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[41734]: osdmap e112: 8 total, 8 up, 8 in 2024-03-24T12:12:47.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418203406' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi006-82076-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:47.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1938122044' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi006-81794-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:47.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3261103760' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi006-81775-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:47.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:47 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-9", "mode": "writeback"}]: dispatch 2024-03-24T12:12:48.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[30108]: 21.2 deep-scrub starts 2024-03-24T12:12:48.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[30108]: 21.2 deep-scrub ok 2024-03-24T12:12:48.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi006-82076-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:48.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi006-81794-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:48.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi006-81775-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:48.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[30108]: pgmap v126: 548 pgs: 22 creating+peering, 136 unknown, 390 active+clean; 169 MiB data, 845 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:12:48.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:12:48.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:48.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:12:48.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:48.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:12:48.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-9", "mode": "writeback"}]': finished 2024-03-24T12:12:48.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi006-81794-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:48.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi006-81775-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:48.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:12:48.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[30108]: osdmap e113: 8 total, 8 up, 8 in 2024-03-24T12:12:48.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1869995198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82043-10", "tierpool":"test-rados-api-smithi006-82043-10-cache", "force_nonempty":""}]: dispatch 2024-03-24T12:12:48.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:12:48.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82043-10", "tierpool":"test-rados-api-smithi006-82043-10-cache", "force_nonempty":""}]: dispatch 2024-03-24T12:12:48.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[41734]: 21.2 deep-scrub starts 2024-03-24T12:12:48.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[41734]: 21.2 deep-scrub ok 2024-03-24T12:12:48.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi006-82076-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:48.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi006-81794-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:48.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi006-81775-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:48.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[41734]: pgmap v126: 548 pgs: 22 creating+peering, 136 unknown, 390 active+clean; 169 MiB data, 845 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:12:48.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:12:48.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:48.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:12:48.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:48.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:12:48.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-9", "mode": "writeback"}]': finished 2024-03-24T12:12:48.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi006-81794-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:48.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi006-81775-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:48.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:12:48.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[41734]: osdmap e113: 8 total, 8 up, 8 in 2024-03-24T12:12:48.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1869995198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82043-10", "tierpool":"test-rados-api-smithi006-82043-10-cache", "force_nonempty":""}]: dispatch 2024-03-24T12:12:48.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:12:48.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:48 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82043-10", "tierpool":"test-rados-api-smithi006-82043-10-cache", "force_nonempty":""}]: dispatch 2024-03-24T12:12:48.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:48 smithi090 ceph-mon[33646]: 21.2 deep-scrub starts 2024-03-24T12:12:48.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:48 smithi090 ceph-mon[33646]: 21.2 deep-scrub ok 2024-03-24T12:12:48.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:48 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi006-82076-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:48.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:48 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi006-81794-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:48.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:48 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi006-81775-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:48.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:48 smithi090 ceph-mon[33646]: pgmap v126: 548 pgs: 22 creating+peering, 136 unknown, 390 active+clean; 169 MiB data, 845 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:12:48.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:48 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:12:48.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:48 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:48.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:48 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:12:48.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:48.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:48 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:12:48.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:48 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-9", "mode": "writeback"}]': finished 2024-03-24T12:12:48.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:48 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi006-81794-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:48.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:48 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi006-81775-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:48.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:12:48.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:48 smithi090 ceph-mon[33646]: osdmap e113: 8 total, 8 up, 8 in 2024-03-24T12:12:48.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1869995198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82043-10", "tierpool":"test-rados-api-smithi006-82043-10-cache", "force_nonempty":""}]: dispatch 2024-03-24T12:12:48.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:48 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:12:48.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:48 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82043-10", "tierpool":"test-rados-api-smithi006-82043-10-cache", "force_nonempty":""}]: dispatch 2024-03-24T12:12:49.878 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:49 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:49.879 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:49 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi006-82076-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]': finished 2024-03-24T12:12:49.879 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:49 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:12:49.879 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:49 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82043-10", "tierpool":"test-rados-api-smithi006-82043-10-cache", "force_nonempty":""}]': finished 2024-03-24T12:12:49.879 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:49 smithi090 ceph-mon[33646]: osdmap e114: 8 total, 8 up, 8 in 2024-03-24T12:12:49.879 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:49 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-9"}]: dispatch 2024-03-24T12:12:49.879 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:49 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1869995198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82043-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:49.879 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:49 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-9"}]: dispatch 2024-03-24T12:12:49.879 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:49 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82043-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:49.879 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:49 smithi090 ceph-mon[33646]: pgmap v129: 588 pgs: 168 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-03-24T12:12:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:49 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:49 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi006-82076-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]': finished 2024-03-24T12:12:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:49 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:12:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:49 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82043-10", "tierpool":"test-rados-api-smithi006-82043-10-cache", "force_nonempty":""}]': finished 2024-03-24T12:12:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:49 smithi006 ceph-mon[30108]: osdmap e114: 8 total, 8 up, 8 in 2024-03-24T12:12:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:49 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-9"}]: dispatch 2024-03-24T12:12:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:49 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1869995198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82043-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:49 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-9"}]: dispatch 2024-03-24T12:12:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:49 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82043-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:49 smithi006 ceph-mon[30108]: pgmap v129: 588 pgs: 168 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-03-24T12:12:49.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:49 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:49.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:49 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi006-82076-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]': finished 2024-03-24T12:12:49.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:49 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:12:49.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:49 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82043-10", "tierpool":"test-rados-api-smithi006-82043-10-cache", "force_nonempty":""}]': finished 2024-03-24T12:12:49.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:49 smithi006 ceph-mon[41734]: osdmap e114: 8 total, 8 up, 8 in 2024-03-24T12:12:49.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:49 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-9"}]: dispatch 2024-03-24T12:12:49.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:49 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1869995198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82043-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:49.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:49 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-9"}]: dispatch 2024-03-24T12:12:49.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:49 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82043-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:49.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:49 smithi006 ceph-mon[41734]: pgmap v129: 588 pgs: 168 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-03-24T12:12:50.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:50 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:50.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:50 smithi006 ceph-mon[30108]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:50.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:50 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:12:50.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:50 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-9"}]': finished 2024-03-24T12:12:50.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:50 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82043-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:50.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:50 smithi006 ceph-mon[30108]: osdmap e115: 8 total, 8 up, 8 in 2024-03-24T12:12:50.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:50 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/682317936' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi006-81775-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:50.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:50 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1869995198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82043-10", "tierpool":"test-rados-api-smithi006-82043-10-cache"}]: dispatch 2024-03-24T12:12:50.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:50 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2899584545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi006-81794-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:50.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:50 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82043-10", "tierpool":"test-rados-api-smithi006-82043-10-cache"}]: dispatch 2024-03-24T12:12:50.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:50 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:50.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:50 smithi006 ceph-mon[41734]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:50.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:50 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:12:50.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:50 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-9"}]': finished 2024-03-24T12:12:50.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:50 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82043-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:50.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:50 smithi006 ceph-mon[41734]: osdmap e115: 8 total, 8 up, 8 in 2024-03-24T12:12:50.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:50 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/682317936' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi006-81775-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:50.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:50 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1869995198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82043-10", "tierpool":"test-rados-api-smithi006-82043-10-cache"}]: dispatch 2024-03-24T12:12:50.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:50 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2899584545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi006-81794-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:50.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:50 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82043-10", "tierpool":"test-rados-api-smithi006-82043-10-cache"}]: dispatch 2024-03-24T12:12:50.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:50 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:50.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:50 smithi090 ceph-mon[33646]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:50.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:50 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:12:50.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:50 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-9"}]': finished 2024-03-24T12:12:50.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:50 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82043-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:50.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:50 smithi090 ceph-mon[33646]: osdmap e115: 8 total, 8 up, 8 in 2024-03-24T12:12:50.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:50 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/682317936' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi006-81775-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:50.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:50 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1869995198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82043-10", "tierpool":"test-rados-api-smithi006-82043-10-cache"}]: dispatch 2024-03-24T12:12:50.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:50 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2899584545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi006-81794-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:50.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:50 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82043-10", "tierpool":"test-rados-api-smithi006-82043-10-cache"}]: dispatch 2024-03-24T12:12:51.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:51 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:51.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:51 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/682317936' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi006-81775-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:51.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:51 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2899584545' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi006-81794-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:51.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:51 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82043-10", "tierpool":"test-rados-api-smithi006-82043-10-cache"}]': finished 2024-03-24T12:12:51.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:51 smithi090 ceph-mon[33646]: osdmap e116: 8 total, 8 up, 8 in 2024-03-24T12:12:51.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:51 smithi090 ceph-mon[33646]: 21.0 deep-scrub starts 2024-03-24T12:12:51.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:51 smithi090 ceph-mon[33646]: 21.0 deep-scrub ok 2024-03-24T12:12:51.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:51 smithi090 ceph-mon[33646]: pgmap v132: 588 pgs: 200 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 12 MiB/s wr, 4 op/s 2024-03-24T12:12:51.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:51 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:51.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:51 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:51.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:51.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/682317936' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi006-81775-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:51.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2899584545' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi006-81794-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:51.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:51 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82043-10", "tierpool":"test-rados-api-smithi006-82043-10-cache"}]': finished 2024-03-24T12:12:51.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:51 smithi006 ceph-mon[30108]: osdmap e116: 8 total, 8 up, 8 in 2024-03-24T12:12:51.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:51 smithi006 ceph-mon[30108]: 21.0 deep-scrub starts 2024-03-24T12:12:51.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:51 smithi006 ceph-mon[30108]: 21.0 deep-scrub ok 2024-03-24T12:12:51.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:51 smithi006 ceph-mon[30108]: pgmap v132: 588 pgs: 200 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 12 MiB/s wr, 4 op/s 2024-03-24T12:12:51.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:51 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:51.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:51 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:51.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:51 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:51.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:51 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/682317936' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi006-81775-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:51.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:51 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2899584545' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi006-81794-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:51.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:51 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82043-10", "tierpool":"test-rados-api-smithi006-82043-10-cache"}]': finished 2024-03-24T12:12:51.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:51 smithi006 ceph-mon[41734]: osdmap e116: 8 total, 8 up, 8 in 2024-03-24T12:12:51.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:51 smithi006 ceph-mon[41734]: 21.0 deep-scrub starts 2024-03-24T12:12:51.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:51 smithi006 ceph-mon[41734]: 21.0 deep-scrub ok 2024-03-24T12:12:51.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:51 smithi006 ceph-mon[41734]: pgmap v132: 588 pgs: 200 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 12 MiB/s wr, 4 op/s 2024-03-24T12:12:51.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:51 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:51.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:51 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:52.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:12:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:12:52.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: Running main() from gmock_main.cc 2024-03-24T12:12:52.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-03-24T12:12:52.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [----------] Global test environment set-up. 2024-03-24T12:12:52.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-03-24T12:12:52.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-03-24T12:12:52.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [ OK ] LibRadosPools.PoolList (11688 ms) 2024-03-24T12:12:52.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-03-24T12:12:52.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (5618 ms) 2024-03-24T12:12:52.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-03-24T12:12:52.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (4266 ms) 2024-03-24T12:12:52.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-03-24T12:12:52.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (2749 ms) 2024-03-24T12:12:52.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-03-24T12:12:52.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (4163 ms) 2024-03-24T12:12:52.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-03-24T12:12:52.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (5018 ms) 2024-03-24T12:12:52.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-03-24T12:12:52.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (5027 ms) 2024-03-24T12:12:52.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-03-24T12:12:52.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (4920 ms) 2024-03-24T12:12:52.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-03-24T12:12:52.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (7932 ms) 2024-03-24T12:12:52.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [----------] 9 tests from LibRadosPools (51382 ms total) 2024-03-24T12:12:52.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: 2024-03-24T12:12:52.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [----------] Global test environment tear-down 2024-03-24T12:12:52.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (51382 ms total) 2024-03-24T12:12:52.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_pool: [ PASSED ] 9 tests. 2024-03-24T12:12:52.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:52 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:52.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:52 smithi090 ceph-mon[33646]: osdmap e117: 8 total, 8 up, 8 in 2024-03-24T12:12:52.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:52 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:52.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:52 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:52.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:52 smithi090 ceph-mon[33646]: 21.3 deep-scrub starts 2024-03-24T12:12:52.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:52 smithi090 ceph-mon[33646]: 21.3 deep-scrub ok 2024-03-24T12:12:52.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:52 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:52.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:52 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:52.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:52 smithi006 ceph-mon[30108]: osdmap e117: 8 total, 8 up, 8 in 2024-03-24T12:12:52.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:52 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:52.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:52 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:52.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:52 smithi006 ceph-mon[30108]: 21.3 deep-scrub starts 2024-03-24T12:12:52.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:52 smithi006 ceph-mon[30108]: 21.3 deep-scrub ok 2024-03-24T12:12:52.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:52 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:52.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:52 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:52.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:52 smithi006 ceph-mon[41734]: osdmap e117: 8 total, 8 up, 8 in 2024-03-24T12:12:52.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:52 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:52.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:52 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:52.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:52 smithi006 ceph-mon[41734]: 21.3 deep-scrub starts 2024-03-24T12:12:52.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:52 smithi006 ceph-mon[41734]: 21.3 deep-scrub ok 2024-03-24T12:12:52.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:52 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:53.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:53 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:12:53.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:53 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:53.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:53 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:53.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:53 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:53.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:53 smithi090 ceph-mon[33646]: osdmap e118: 8 total, 8 up, 8 in 2024-03-24T12:12:53.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:53 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1386638036' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi006-81775-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:53.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:53 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2160000289' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi006-81794-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:53.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:53 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi006-81794-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:53.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:53 smithi090 ceph-mon[33646]: 21.1 deep-scrub starts 2024-03-24T12:12:53.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:53 smithi090 ceph-mon[33646]: 21.1 deep-scrub ok 2024-03-24T12:12:53.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:53 smithi090 ceph-mon[33646]: pgmap v135: 556 pgs: 64 creating+peering, 96 unknown, 396 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-03-24T12:12:53.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:53 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:12:53.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:53 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:12:53.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:53 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:53.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:12:53.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:53.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:53.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:53.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[30108]: osdmap e118: 8 total, 8 up, 8 in 2024-03-24T12:12:53.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1386638036' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi006-81775-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:53.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2160000289' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi006-81794-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:53.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi006-81794-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:53.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[30108]: 21.1 deep-scrub starts 2024-03-24T12:12:53.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[30108]: 21.1 deep-scrub ok 2024-03-24T12:12:53.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[30108]: pgmap v135: 556 pgs: 64 creating+peering, 96 unknown, 396 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-03-24T12:12:53.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:12:53.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:12:53.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:53.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:12:53.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:53.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:53.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:53.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[41734]: osdmap e118: 8 total, 8 up, 8 in 2024-03-24T12:12:53.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1386638036' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi006-81775-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:53.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2160000289' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi006-81794-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:53.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi006-81794-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:53.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[41734]: 21.1 deep-scrub starts 2024-03-24T12:12:53.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[41734]: 21.1 deep-scrub ok 2024-03-24T12:12:53.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[41734]: pgmap v135: 556 pgs: 64 creating+peering, 96 unknown, 396 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-03-24T12:12:53.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:12:53.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:12:53.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:53 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:12:54.842 INFO:tasks.workunit.client.0.smithi006.stdout: misc: Running main() from gmock_main.cc 2024-03-24T12:12:54.842 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [==========] Running 12 tests from 1 test suite. 2024-03-24T12:12:54.842 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [----------] Global test environment set-up. 2024-03-24T12:12:54.842 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [----------] 12 tests from NeoRadosMisc 2024-03-24T12:12:54.842 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ RUN ] NeoRadosMisc.Version 2024-03-24T12:12:54.842 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ OK ] NeoRadosMisc.Version (10682 ms) 2024-03-24T12:12:54.842 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ RUN ] NeoRadosMisc.WaitOSDMap 2024-03-24T12:12:54.842 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ OK ] NeoRadosMisc.WaitOSDMap (5743 ms) 2024-03-24T12:12:54.842 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ RUN ] NeoRadosMisc.LongName 2024-03-24T12:12:54.842 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ OK ] NeoRadosMisc.LongName (7013 ms) 2024-03-24T12:12:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ RUN ] NeoRadosMisc.LongLocator 2024-03-24T12:12:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ OK ] NeoRadosMisc.LongLocator (4681 ms) 2024-03-24T12:12:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ RUN ] NeoRadosMisc.LongNamespace 2024-03-24T12:12:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ OK ] NeoRadosMisc.LongNamespace (3984 ms) 2024-03-24T12:12:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ RUN ] NeoRadosMisc.LongAttrName 2024-03-24T12:12:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ OK ] NeoRadosMisc.LongAttrName (2998 ms) 2024-03-24T12:12:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ RUN ] NeoRadosMisc.Exec 2024-03-24T12:12:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ OK ] NeoRadosMisc.Exec (3923 ms) 2024-03-24T12:12:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ RUN ] NeoRadosMisc.Operate1 2024-03-24T12:12:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ OK ] NeoRadosMisc.Operate1 (3068 ms) 2024-03-24T12:12:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ RUN ] NeoRadosMisc.Operate2 2024-03-24T12:12:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ OK ] NeoRadosMisc.Operate2 (2938 ms) 2024-03-24T12:12:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ RUN ] NeoRadosMisc.BigObject 2024-03-24T12:12:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ OK ] NeoRadosMisc.BigObject (3008 ms) 2024-03-24T12:12:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ RUN ] NeoRadosMisc.BigAttr 2024-03-24T12:12:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ OK ] NeoRadosMisc.BigAttr (1999 ms) 2024-03-24T12:12:54.844 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ RUN ] NeoRadosMisc.WriteSame 2024-03-24T12:12:54.844 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ OK ] NeoRadosMisc.WriteSame (3015 ms) 2024-03-24T12:12:54.844 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [----------] 12 tests from NeoRadosMisc (53052 ms total) 2024-03-24T12:12:54.844 INFO:tasks.workunit.client.0.smithi006.stdout: misc: 2024-03-24T12:12:54.844 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [----------] Global test environment tear-down 2024-03-24T12:12:54.844 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [==========] 12 tests from 1 test suite ran. (53052 ms total) 2024-03-24T12:12:54.844 INFO:tasks.workunit.client.0.smithi006.stdout: misc: [ PASSED ] 12 tests. 2024-03-24T12:12:54.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:54.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1386638036' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi006-81775-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:54.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:54 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi006-81794-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:54.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:54 smithi090 ceph-mon[33646]: osdmap e119: 8 total, 8 up, 8 in 2024-03-24T12:12:54.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:54.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:54 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:54.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:54.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1386638036' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi006-81775-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:54.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:54 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi006-81794-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:54.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:54 smithi006 ceph-mon[30108]: osdmap e119: 8 total, 8 up, 8 in 2024-03-24T12:12:54.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:54.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:54 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:54.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:54.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1386638036' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi006-81775-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:54.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:54 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi006-81794-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:54.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:54 smithi006 ceph-mon[41734]: osdmap e119: 8 total, 8 up, 8 in 2024-03-24T12:12:54.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:54.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:54 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:12:55.655 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:12:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:12:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:12:55.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:55 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:55.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:55 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-11", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:12:55.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:55 smithi090 ceph-mon[33646]: osdmap e120: 8 total, 8 up, 8 in 2024-03-24T12:12:55.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:55 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-11"}]: dispatch 2024-03-24T12:12:55.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:55 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-11"}]: dispatch 2024-03-24T12:12:55.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:55 smithi090 ceph-mon[33646]: pgmap v138: 492 pgs: 32 creating+peering, 96 unknown, 364 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 6 op/s 2024-03-24T12:12:55.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:55 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:55.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:55 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-11", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:12:55.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:55 smithi006 ceph-mon[30108]: osdmap e120: 8 total, 8 up, 8 in 2024-03-24T12:12:55.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:55 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-11"}]: dispatch 2024-03-24T12:12:55.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:55 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-11"}]: dispatch 2024-03-24T12:12:55.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:55 smithi006 ceph-mon[30108]: pgmap v138: 492 pgs: 32 creating+peering, 96 unknown, 364 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 6 op/s 2024-03-24T12:12:55.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:55 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:55.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:55 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-11", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:12:55.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:55 smithi006 ceph-mon[41734]: osdmap e120: 8 total, 8 up, 8 in 2024-03-24T12:12:55.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:55 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-11"}]: dispatch 2024-03-24T12:12:55.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:55 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-11"}]: dispatch 2024-03-24T12:12:55.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:55 smithi006 ceph-mon[41734]: pgmap v138: 492 pgs: 32 creating+peering, 96 unknown, 364 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 6 op/s 2024-03-24T12:12:56.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:56 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:56.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:56 smithi090 ceph-mon[33646]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:56.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:56 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-11"}]': finished 2024-03-24T12:12:56.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:56 smithi090 ceph-mon[33646]: osdmap e121: 8 total, 8 up, 8 in 2024-03-24T12:12:56.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:56 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-11", "mode": "writeback"}]: dispatch 2024-03-24T12:12:56.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:56 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2813121178' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi006-81794-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:56.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:56 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2210012240' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi006-81775-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:56.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:56 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-11", "mode": "writeback"}]: dispatch 2024-03-24T12:12:56.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:56 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi006-81794-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:56.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:56 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi006-81775-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:56.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:56 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:56.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:56 smithi006 ceph-mon[30108]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:56.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:56 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-11"}]': finished 2024-03-24T12:12:56.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:56 smithi006 ceph-mon[30108]: osdmap e121: 8 total, 8 up, 8 in 2024-03-24T12:12:56.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:56 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-11", "mode": "writeback"}]: dispatch 2024-03-24T12:12:56.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:56 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2813121178' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi006-81794-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:56.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:56 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2210012240' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi006-81775-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:56.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:56 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-11", "mode": "writeback"}]: dispatch 2024-03-24T12:12:56.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:56 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi006-81794-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:56.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:56 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi006-81775-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:56.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:56 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:56.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:56 smithi006 ceph-mon[41734]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:12:56.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:56 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-11"}]': finished 2024-03-24T12:12:56.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:56 smithi006 ceph-mon[41734]: osdmap e121: 8 total, 8 up, 8 in 2024-03-24T12:12:56.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:56 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-11", "mode": "writeback"}]: dispatch 2024-03-24T12:12:56.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:56 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2813121178' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi006-81794-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:56.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:56 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2210012240' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi006-81775-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:56.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:56 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-11", "mode": "writeback"}]: dispatch 2024-03-24T12:12:56.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:56 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi006-81794-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:56.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:56 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi006-81775-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:57.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:57 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:57.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:57 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:12:57.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:57 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-11", "mode": "writeback"}]': finished 2024-03-24T12:12:57.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:57 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi006-81794-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:57.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:57 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi006-81775-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:57.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:57 smithi090 ceph-mon[33646]: osdmap e122: 8 total, 8 up, 8 in 2024-03-24T12:12:57.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:57 smithi090 ceph-mon[33646]: pgmap v141: 524 pgs: 28 creating+peering, 93 unknown, 403 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-03-24T12:12:57.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:57 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:57.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:57 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:12:57.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:57 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-11", "mode": "writeback"}]': finished 2024-03-24T12:12:57.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:57 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi006-81794-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:57.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:57 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi006-81775-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:57.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:57 smithi006 ceph-mon[30108]: osdmap e122: 8 total, 8 up, 8 in 2024-03-24T12:12:57.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:57 smithi006 ceph-mon[30108]: pgmap v141: 524 pgs: 28 creating+peering, 93 unknown, 403 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-03-24T12:12:57.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:57 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:57.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:57 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:12:57.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:57 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-11", "mode": "writeback"}]': finished 2024-03-24T12:12:57.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:57 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi006-81794-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:57.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:57 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi006-81775-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:12:57.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:57 smithi006 ceph-mon[41734]: osdmap e122: 8 total, 8 up, 8 in 2024-03-24T12:12:57.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:57 smithi006 ceph-mon[41734]: pgmap v141: 524 pgs: 28 creating+peering, 93 unknown, 403 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-03-24T12:12:58.823 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: Running main() from gmock_main.cc 2024-03-24T12:12:58.823 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [==========] Running 14 tests from 1 test suite. 2024-03-24T12:12:58.823 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [----------] Global test environment set-up. 2024-03-24T12:12:58.823 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps 2024-03-24T12:12:58.823 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ RUN ] NeoRadosReadOps.SetOpFlags 2024-03-24T12:12:58.823 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ OK ] NeoRadosReadOps.SetOpFlags (11362 ms) 2024-03-24T12:12:58.823 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertExists 2024-03-24T12:12:58.823 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertExists (8485 ms) 2024-03-24T12:12:58.823 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertVersion 2024-03-24T12:12:58.824 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertVersion (3591 ms) 2024-03-24T12:12:58.824 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpXattr 2024-03-24T12:12:58.824 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpXattr (2679 ms) 2024-03-24T12:12:58.824 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ RUN ] NeoRadosReadOps.Read 2024-03-24T12:12:58.824 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ OK ] NeoRadosReadOps.Read (2868 ms) 2024-03-24T12:12:58.824 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ RUN ] NeoRadosReadOps.Checksum 2024-03-24T12:12:58.824 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ OK ] NeoRadosReadOps.Checksum (3126 ms) 2024-03-24T12:12:58.824 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ RUN ] NeoRadosReadOps.RWOrderedRead 2024-03-24T12:12:58.824 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ OK ] NeoRadosReadOps.RWOrderedRead (4064 ms) 2024-03-24T12:12:58.824 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ RUN ] NeoRadosReadOps.ShortRead 2024-03-24T12:12:58.824 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ OK ] NeoRadosReadOps.ShortRead (2842 ms) 2024-03-24T12:12:58.824 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ RUN ] NeoRadosReadOps.Exec 2024-03-24T12:12:58.824 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ OK ] NeoRadosReadOps.Exec (3073 ms) 2024-03-24T12:12:58.825 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ RUN ] NeoRadosReadOps.Stat 2024-03-24T12:12:58.825 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ OK ] NeoRadosReadOps.Stat (2947 ms) 2024-03-24T12:12:58.825 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ RUN ] NeoRadosReadOps.Omap 2024-03-24T12:12:58.825 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ OK ] NeoRadosReadOps.Omap (2999 ms) 2024-03-24T12:12:58.825 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ RUN ] NeoRadosReadOps.OmapNuls 2024-03-24T12:12:58.825 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ OK ] NeoRadosReadOps.OmapNuls (3018 ms) 2024-03-24T12:12:58.825 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ RUN ] NeoRadosReadOps.GetXattrs 2024-03-24T12:12:58.825 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ OK ] NeoRadosReadOps.GetXattrs (2973 ms) 2024-03-24T12:12:58.825 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpExt 2024-03-24T12:12:58.825 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpExt (3009 ms) 2024-03-24T12:12:58.825 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps (57037 ms total) 2024-03-24T12:12:58.825 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: 2024-03-24T12:12:58.826 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [----------] Global test environment tear-down 2024-03-24T12:12:58.826 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [==========] 14 tests from 1 test suite ran. (57037 ms total) 2024-03-24T12:12:58.826 INFO:tasks.workunit.client.0.smithi006.stdout: read_operations: [ PASSED ] 14 tests. 2024-03-24T12:12:58.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:58 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:58.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:58 smithi090 ceph-mon[33646]: osdmap e123: 8 total, 8 up, 8 in 2024-03-24T12:12:58.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:58 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:12:58.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:58 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:12:58.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:58 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:58.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:58 smithi006 ceph-mon[30108]: osdmap e123: 8 total, 8 up, 8 in 2024-03-24T12:12:58.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:58 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:12:58.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:58 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:12:58.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:58 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:58.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:58 smithi006 ceph-mon[41734]: osdmap e123: 8 total, 8 up, 8 in 2024-03-24T12:12:58.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:58 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:12:58.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:58 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:12:59.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:59 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:59.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:59 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:12:59.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:59 smithi090 ceph-mon[33646]: osdmap e124: 8 total, 8 up, 8 in 2024-03-24T12:12:59.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:59 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-11"}]: dispatch 2024-03-24T12:12:59.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:59 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-11"}]: dispatch 2024-03-24T12:12:59.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:59 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418203406' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:59.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:59 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3735076754' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi006-81794-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:59.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:59 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:59.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:59 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3022802190' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi006-81775-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:59.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:12:59 smithi090 ceph-mon[33646]: pgmap v144: 516 pgs: 96 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 260 KiB/s wr, 9 op/s 2024-03-24T12:12:59.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:59 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:59.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:59 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:12:59.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:59 smithi006 ceph-mon[30108]: osdmap e124: 8 total, 8 up, 8 in 2024-03-24T12:12:59.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:59 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-11"}]: dispatch 2024-03-24T12:12:59.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:59 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-11"}]: dispatch 2024-03-24T12:12:59.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:59 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418203406' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:59.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:59 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3735076754' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi006-81794-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:59.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:59 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:59.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:59 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3022802190' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi006-81775-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:59.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:12:59 smithi006 ceph-mon[30108]: pgmap v144: 516 pgs: 96 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 260 KiB/s wr, 9 op/s 2024-03-24T12:12:59.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:59 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:12:59.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:59 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:12:59.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:59 smithi006 ceph-mon[41734]: osdmap e124: 8 total, 8 up, 8 in 2024-03-24T12:12:59.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:59 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-11"}]: dispatch 2024-03-24T12:12:59.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:59 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-11"}]: dispatch 2024-03-24T12:12:59.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:59 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418203406' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:59.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:59 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3735076754' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi006-81794-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:59.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:59 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:12:59.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:59 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3022802190' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi006-81775-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:12:59.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:12:59 smithi006 ceph-mon[41734]: pgmap v144: 516 pgs: 96 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 260 KiB/s wr, 9 op/s 2024-03-24T12:13:00.932 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:00 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:00.932 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:00 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:13:00.932 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:00 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-11"}]': finished 2024-03-24T12:13:00.932 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:00 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3735076754' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi006-81794-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:00.932 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:00 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]': finished 2024-03-24T12:13:00.932 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:00 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3022802190' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi006-81775-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:00.932 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:00 smithi090 ceph-mon[33646]: osdmap e125: 8 total, 8 up, 8 in 2024-03-24T12:13:00.932 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:00 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418203406' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:13:00.932 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:00 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:13:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:00 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:00 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:13:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:00 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-11"}]': finished 2024-03-24T12:13:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:00 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3735076754' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi006-81794-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:00 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]': finished 2024-03-24T12:13:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:00 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3022802190' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi006-81775-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:00 smithi006 ceph-mon[30108]: osdmap e125: 8 total, 8 up, 8 in 2024-03-24T12:13:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:00 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418203406' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:13:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:00 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:13:00.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:00 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:00.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:00 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:13:00.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:00 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-11"}]': finished 2024-03-24T12:13:00.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:00 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3735076754' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi006-81794-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:00.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:00 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi006-82076-10"}]': finished 2024-03-24T12:13:00.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:00 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3022802190' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi006-81775-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:00.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:00 smithi006 ceph-mon[41734]: osdmap e125: 8 total, 8 up, 8 in 2024-03-24T12:13:00.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:00 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418203406' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:13:00.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:00 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi006-82076-10"}]: dispatch 2024-03-24T12:13:01.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:01.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:01 smithi090 ceph-mon[33646]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:01.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi006-82076-10"}]': finished 2024-03-24T12:13:01.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:01 smithi090 ceph-mon[33646]: osdmap e126: 8 total, 8 up, 8 in 2024-03-24T12:13:01.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2037860533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:01.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:01 smithi090 ceph-mon[33646]: pgmap v147: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 259 KiB/s wr, 5 op/s 2024-03-24T12:13:01.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:01.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2037860533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:01.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:01.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2037860533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:01.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:01.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:01.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:01 smithi006 ceph-mon[30108]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:01.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi006-82076-10"}]': finished 2024-03-24T12:13:01.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:01 smithi006 ceph-mon[30108]: osdmap e126: 8 total, 8 up, 8 in 2024-03-24T12:13:01.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2037860533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:01.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:01 smithi006 ceph-mon[30108]: pgmap v147: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 259 KiB/s wr, 5 op/s 2024-03-24T12:13:01.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:01.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2037860533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:01.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:01.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2037860533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:01.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:01.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:01.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:01 smithi006 ceph-mon[41734]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:01.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi006-82076-10"}]': finished 2024-03-24T12:13:01.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:01 smithi006 ceph-mon[41734]: osdmap e126: 8 total, 8 up, 8 in 2024-03-24T12:13:01.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2037860533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:01.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:01 smithi006 ceph-mon[41734]: pgmap v147: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 259 KiB/s wr, 5 op/s 2024-03-24T12:13:01.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:01.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2037860533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:01.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:01.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2037860533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:01.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:02.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:13:02 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:13:02.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:02 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:02.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:02 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:02.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:02 smithi090 ceph-mon[33646]: osdmap e127: 8 total, 8 up, 8 in 2024-03-24T12:13:02.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:02 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2037860533' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi006-82076-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:02.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:02 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:02.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:02 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2628121253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi006-81794-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:02.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:02 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4053892838' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi006-81775-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:02.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:02 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi006-82076-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:02.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:02 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:02.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:02 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi006-81794-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:02.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:02 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi006-81775-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:02.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:02 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:13:02.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:02 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:13:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[30108]: osdmap e127: 8 total, 8 up, 8 in 2024-03-24T12:13:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2037860533' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi006-82076-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2628121253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi006-81794-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4053892838' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi006-81775-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi006-82076-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:02.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:02.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi006-81794-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:02.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi006-81775-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:02.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:13:02.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:13:02.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:02.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:02.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[41734]: osdmap e127: 8 total, 8 up, 8 in 2024-03-24T12:13:02.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2037860533' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi006-82076-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:02.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:02.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2628121253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi006-81794-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:02.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4053892838' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi006-81775-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:02.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi006-82076-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:02.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:02.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi006-81794-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:02.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi006-81775-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:02.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:13:02.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:02 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:13:03.096 INFO:tasks.workunit.client.0.smithi006.stdout: io: Running main() from gmock_main.cc 2024-03-24T12:13:03.097 INFO:tasks.workunit.client.0.smithi006.stdout: io: [==========] Running 14 tests from 1 test suite. 2024-03-24T12:13:03.097 INFO:tasks.workunit.client.0.smithi006.stdout: io: [----------] Global test environment set-up. 2024-03-24T12:13:03.097 INFO:tasks.workunit.client.0.smithi006.stdout: io: [----------] 14 tests from NeoRadosIo 2024-03-24T12:13:03.097 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ RUN ] NeoRadosIo.Limits 2024-03-24T12:13:03.097 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ OK ] NeoRadosIo.Limits (12314 ms) 2024-03-24T12:13:03.097 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ RUN ] NeoRadosIo.SimpleWrite 2024-03-24T12:13:03.097 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ OK ] NeoRadosIo.SimpleWrite (10108 ms) 2024-03-24T12:13:03.097 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ RUN ] NeoRadosIo.ReadOp 2024-03-24T12:13:03.097 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ OK ] NeoRadosIo.ReadOp (3770 ms) 2024-03-24T12:13:03.097 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ RUN ] NeoRadosIo.SparseRead 2024-03-24T12:13:03.097 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ OK ] NeoRadosIo.SparseRead (4930 ms) 2024-03-24T12:13:03.097 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ RUN ] NeoRadosIo.RoundTrip 2024-03-24T12:13:03.097 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ OK ] NeoRadosIo.RoundTrip (2981 ms) 2024-03-24T12:13:03.097 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-03-24T12:13:03.097 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ OK ] NeoRadosIo.ReadIntoBuufferlist (3155 ms) 2024-03-24T12:13:03.098 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-03-24T12:13:03.098 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ OK ] NeoRadosIo.OverlappingWriteRoundTrip (2843 ms) 2024-03-24T12:13:03.098 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-03-24T12:13:03.098 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ OK ] NeoRadosIo.WriteFullRoundTrip (3084 ms) 2024-03-24T12:13:03.098 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ RUN ] NeoRadosIo.AppendRoundTrip 2024-03-24T12:13:03.098 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ OK ] NeoRadosIo.AppendRoundTrip (3196 ms) 2024-03-24T12:13:03.098 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ RUN ] NeoRadosIo.Trunc 2024-03-24T12:13:03.098 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ OK ] NeoRadosIo.Trunc (2729 ms) 2024-03-24T12:13:03.098 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ RUN ] NeoRadosIo.Remove 2024-03-24T12:13:03.098 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ OK ] NeoRadosIo.Remove (3024 ms) 2024-03-24T12:13:03.099 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ RUN ] NeoRadosIo.XattrsRoundTrip 2024-03-24T12:13:03.099 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ OK ] NeoRadosIo.XattrsRoundTrip (2974 ms) 2024-03-24T12:13:03.099 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ RUN ] NeoRadosIo.RmXattr 2024-03-24T12:13:03.099 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ OK ] NeoRadosIo.RmXattr (3018 ms) 2024-03-24T12:13:03.099 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ RUN ] NeoRadosIo.GetXattrs 2024-03-24T12:13:03.099 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ OK ] NeoRadosIo.GetXattrs (3262 ms) 2024-03-24T12:13:03.099 INFO:tasks.workunit.client.0.smithi006.stdout: io: [----------] 14 tests from NeoRadosIo (61391 ms total) 2024-03-24T12:13:03.099 INFO:tasks.workunit.client.0.smithi006.stdout: io: 2024-03-24T12:13:03.099 INFO:tasks.workunit.client.0.smithi006.stdout: io: [----------] Global test environment tear-down 2024-03-24T12:13:03.099 INFO:tasks.workunit.client.0.smithi006.stdout: io: [==========] 14 tests from 1 test suite ran. (61391 ms total) 2024-03-24T12:13:03.099 INFO:tasks.workunit.client.0.smithi006.stdout: io: [ PASSED ] 14 tests. 2024-03-24T12:13:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:03 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:03 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi006-81794-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:03 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi006-81775-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:03 smithi090 ceph-mon[33646]: pgmap v150: 484 pgs: 96 creating+peering, 32 unknown, 356 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-03-24T12:13:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:03 smithi090 ceph-mon[33646]: osdmap e128: 8 total, 8 up, 8 in 2024-03-24T12:13:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:03 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:03 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi006-81794-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:03 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi006-81775-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:03 smithi006 ceph-mon[30108]: pgmap v150: 484 pgs: 96 creating+peering, 32 unknown, 356 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-03-24T12:13:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:03 smithi006 ceph-mon[30108]: osdmap e128: 8 total, 8 up, 8 in 2024-03-24T12:13:03.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:03.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:03 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:03.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:03 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi006-81794-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:03.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:03 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi006-81775-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:03.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:03 smithi006 ceph-mon[41734]: pgmap v150: 484 pgs: 96 creating+peering, 32 unknown, 356 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-03-24T12:13:03.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:03 smithi006 ceph-mon[41734]: osdmap e128: 8 total, 8 up, 8 in 2024-03-24T12:13:04.461 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-03-24T12:13:04.461 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [----------] Global test environment set-up. 2024-03-24T12:13:04.462 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-03-24T12:13:04.462 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-03-24T12:13:04.462 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-03-24T12:13:04.462 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-03-24T12:13:04.462 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: 2024-03-24T12:13:04.462 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-03-24T12:13:04.462 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-03-24T12:13:04.462 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-03-24T12:13:04.462 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: 2024-03-24T12:12:01.538+0000 7fc8686d2780 -1 failed for service _ceph-mon._tcp 2024-03-24T12:13:04.462 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: 2024-03-24T12:12:01.538+0000 7fc8686d2780 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-03-24T12:13:04.462 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (94 ms) 2024-03-24T12:13:04.462 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-03-24T12:13:04.462 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2013 ms) 2024-03-24T12:13:04.463 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (2107 ms total) 2024-03-24T12:13:04.463 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: 2024-03-24T12:13:04.463 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-03-24T12:13:04.463 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-03-24T12:13:04.463 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4aa65600 2024-03-24T12:13:04.463 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada2900 2024-03-24T12:13:04.463 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada3250 2024-03-24T12:13:04.463 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada2db0 2024-03-24T12:13:04.463 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada3730 2024-03-24T12:13:04.463 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada41c0 2024-03-24T12:13:04.463 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada3fc0 2024-03-24T12:13:04.463 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada4690 2024-03-24T12:13:04.463 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada4b00 2024-03-24T12:13:04.463 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada4c10 2024-03-24T12:13:04.464 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada50d0 2024-03-24T12:13:04.464 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada5270 2024-03-24T12:13:04.464 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada47a0 2024-03-24T12:13:04.464 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada48b0 2024-03-24T12:13:04.464 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada49c0 2024-03-24T12:13:04.464 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada5380 2024-03-24T12:13:04.464 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada4d20 2024-03-24T12:13:04.464 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada4e30 2024-03-24T12:13:04.464 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada5840 2024-03-24T12:13:04.464 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada5d00 2024-03-24T12:13:04.464 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada5950 2024-03-24T12:13:04.464 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada5a80 2024-03-24T12:13:04.464 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada5e10 2024-03-24T12:13:04.464 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada5f20 2024-03-24T12:13:04.464 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada5bf0 2024-03-24T12:13:04.465 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada6390 2024-03-24T12:13:04.465 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada6030 2024-03-24T12:13:04.465 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada6140 2024-03-24T12:13:04.465 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada6250 2024-03-24T12:13:04.466 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada6970 2024-03-24T12:13:04.466 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada64d0 2024-03-24T12:13:04.466 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada6600 2024-03-24T12:13:04.466 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada6730 2024-03-24T12:13:04.466 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada6860 2024-03-24T12:13:04.466 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada6f40 2024-03-24T12:13:04.466 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada6aa0 2024-03-24T12:13:04.466 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada6bf0 2024-03-24T12:13:04.466 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada6d00 2024-03-24T12:13:04.466 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada6e10 2024-03-24T12:13:04.466 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada7530 2024-03-24T12:13:04.466 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada7050 2024-03-24T12:13:04.466 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada7160 2024-03-24T12:13:04.467 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada7270 2024-03-24T12:13:04.467 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada73a0 2024-03-24T12:13:04.467 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada7b40 2024-03-24T12:13:04.467 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada7660 2024-03-24T12:13:04.467 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada77c0 2024-03-24T12:13:04.467 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada78f0 2024-03-24T12:13:04.467 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada7c50 2024-03-24T12:13:04.467 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada80e0 2024-03-24T12:13:04.467 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada8550 2024-03-24T12:13:04.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:04 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:04.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:04 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi006-82076-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]': finished 2024-03-24T12:13:04.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:04 smithi090 ceph-mon[33646]: osdmap e129: 8 total, 8 up, 8 in 2024-03-24T12:13:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:04 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:04 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi006-82076-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]': finished 2024-03-24T12:13:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:04 smithi006 ceph-mon[30108]: osdmap e129: 8 total, 8 up, 8 in 2024-03-24T12:13:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:04 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:04 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi006-82076-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]': finished 2024-03-24T12:13:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:04 smithi006 ceph-mon[41734]: osdmap e129: 8 total, 8 up, 8 in 2024-03-24T12:13:05.671 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:13:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:13:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:13:05.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:05 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:05.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:05 smithi090 ceph-mon[33646]: osdmap e130: 8 total, 8 up, 8 in 2024-03-24T12:13:05.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:05 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2000000857' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi006-81775-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:05.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:05 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3897008856' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi006-81794-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:05.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:05 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi006-81775-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:05.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:05 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi006-81794-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:05.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:05 smithi090 ceph-mon[33646]: pgmap v153: 492 pgs: 32 creating+peering, 104 unknown, 356 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-24T12:13:05.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:05 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:05.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:05 smithi006 ceph-mon[30108]: osdmap e130: 8 total, 8 up, 8 in 2024-03-24T12:13:05.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:05 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2000000857' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi006-81775-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:05.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:05 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3897008856' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi006-81794-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:05.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:05 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi006-81775-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:05.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:05 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi006-81794-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:05.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:05 smithi006 ceph-mon[30108]: pgmap v153: 492 pgs: 32 creating+peering, 104 unknown, 356 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-24T12:13:05.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:05 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:05.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:05 smithi006 ceph-mon[41734]: osdmap e130: 8 total, 8 up, 8 in 2024-03-24T12:13:05.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:05 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2000000857' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi006-81775-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:05.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:05 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3897008856' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi006-81794-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:05.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:05 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi006-81775-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:05.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:05 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi006-81794-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:05.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:05 smithi006 ceph-mon[41734]: pgmap v153: 492 pgs: 32 creating+peering, 104 unknown, 356 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-24T12:13:06.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:06 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:06.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:06 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:06.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:06 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi006-81775-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:06.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:06 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi006-81794-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:06.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:06 smithi090 ceph-mon[33646]: osdmap e131: 8 total, 8 up, 8 in 2024-03-24T12:13:06.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:06 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:06.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:06 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:06.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:06 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi006-81775-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:06.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:06 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi006-81794-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:06.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:06 smithi006 ceph-mon[30108]: osdmap e131: 8 total, 8 up, 8 in 2024-03-24T12:13:06.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:06 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:06.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:06 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:06.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:06 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi006-81775-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:06.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:06 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi006-81794-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:06.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:06 smithi006 ceph-mon[41734]: osdmap e131: 8 total, 8 up, 8 in 2024-03-24T12:13:07.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:07 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:07.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:07 smithi090 ceph-mon[33646]: osdmap e132: 8 total, 8 up, 8 in 2024-03-24T12:13:07.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:07 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:13:07.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:07 smithi090 ceph-mon[33646]: pgmap v156: 428 pgs: 22 creating+peering, 30 unknown, 376 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 6.2 KiB/s wr, 6 op/s 2024-03-24T12:13:07.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:07 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:13:07.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:07 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:07.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:07 smithi006 ceph-mon[30108]: osdmap e132: 8 total, 8 up, 8 in 2024-03-24T12:13:07.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:07 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:13:07.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:07 smithi006 ceph-mon[30108]: pgmap v156: 428 pgs: 22 creating+peering, 30 unknown, 376 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 6.2 KiB/s wr, 6 op/s 2024-03-24T12:13:07.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:07 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:13:07.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:07 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:07.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:07 smithi006 ceph-mon[41734]: osdmap e132: 8 total, 8 up, 8 in 2024-03-24T12:13:07.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:07 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:13:07.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:07 smithi006 ceph-mon[41734]: pgmap v156: 428 pgs: 22 creating+peering, 30 unknown, 376 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 6.2 KiB/s wr, 6 op/s 2024-03-24T12:13:07.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:07 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:13:08.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:08.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-13", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:13:08.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-13"}]: dispatch 2024-03-24T12:13:08.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:08 smithi090 ceph-mon[33646]: osdmap e133: 8 total, 8 up, 8 in 2024-03-24T12:13:08.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3159912076' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi006-81794-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:08.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/42553659' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi006-81775-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:08.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi006-81794-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:08.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-13"}]: dispatch 2024-03-24T12:13:08.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi006-81775-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:08.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:08.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-13", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:13:08.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-13"}]: dispatch 2024-03-24T12:13:08.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:08 smithi006 ceph-mon[41734]: osdmap e133: 8 total, 8 up, 8 in 2024-03-24T12:13:08.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3159912076' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi006-81794-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:08.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/42553659' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi006-81775-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:08.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi006-81794-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:08.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-13"}]: dispatch 2024-03-24T12:13:08.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi006-81775-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:08.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:08.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-13", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:13:08.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-13"}]: dispatch 2024-03-24T12:13:08.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:08 smithi006 ceph-mon[30108]: osdmap e133: 8 total, 8 up, 8 in 2024-03-24T12:13:08.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3159912076' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi006-81794-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:08.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/42553659' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi006-81775-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:08.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi006-81794-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:08.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-13"}]: dispatch 2024-03-24T12:13:08.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi006-81775-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:09.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:09 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:09.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:09 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi006-81794-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:09.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:09 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-13"}]': finished 2024-03-24T12:13:09.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:09 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi006-81775-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:09.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:09 smithi006 ceph-mon[30108]: osdmap e134: 8 total, 8 up, 8 in 2024-03-24T12:13:09.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:09 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-13", "mode": "writeback"}]: dispatch 2024-03-24T12:13:09.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:09 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-13", "mode": "writeback"}]: dispatch 2024-03-24T12:13:09.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:09 smithi006 ceph-mon[30108]: pgmap v159: 492 pgs: 96 unknown, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 129 KiB/s wr, 126 op/s 2024-03-24T12:13:09.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:09 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:09.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:09 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi006-81794-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:09.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:09 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-13"}]': finished 2024-03-24T12:13:09.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:09 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi006-81775-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:09.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:09 smithi006 ceph-mon[41734]: osdmap e134: 8 total, 8 up, 8 in 2024-03-24T12:13:09.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:09 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-13", "mode": "writeback"}]: dispatch 2024-03-24T12:13:09.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:09 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-13", "mode": "writeback"}]: dispatch 2024-03-24T12:13:09.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:09 smithi006 ceph-mon[41734]: pgmap v159: 492 pgs: 96 unknown, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 129 KiB/s wr, 126 op/s 2024-03-24T12:13:10.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:09 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:10.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:09 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi006-81794-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:10.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:09 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-13"}]': finished 2024-03-24T12:13:10.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:09 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi006-81775-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:10.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:09 smithi090 ceph-mon[33646]: osdmap e134: 8 total, 8 up, 8 in 2024-03-24T12:13:10.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:09 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-13", "mode": "writeback"}]: dispatch 2024-03-24T12:13:10.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:09 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-13", "mode": "writeback"}]: dispatch 2024-03-24T12:13:10.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:09 smithi090 ceph-mon[33646]: pgmap v159: 492 pgs: 96 unknown, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 129 KiB/s wr, 126 op/s 2024-03-24T12:13:10.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:10.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:13:10.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-13", "mode": "writeback"}]': finished 2024-03-24T12:13:10.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: osdmap e135: 8 total, 8 up, 8 in 2024-03-24T12:13:10.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.0"}]: dispatch 2024-03-24T12:13:10.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.0"}]: dispatch 2024-03-24T12:13:10.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.1"}]: dispatch 2024-03-24T12:13:10.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.1"}]: dispatch 2024-03-24T12:13:10.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.2"}]: dispatch 2024-03-24T12:13:10.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.2"}]: dispatch 2024-03-24T12:13:10.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.3"}]: dispatch 2024-03-24T12:13:10.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.3"}]: dispatch 2024-03-24T12:13:10.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.4"}]: dispatch 2024-03-24T12:13:10.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.4"}]: dispatch 2024-03-24T12:13:10.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.5"}]: dispatch 2024-03-24T12:13:10.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.5"}]: dispatch 2024-03-24T12:13:10.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.6"}]: dispatch 2024-03-24T12:13:10.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.6"}]: dispatch 2024-03-24T12:13:10.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.7"}]: dispatch 2024-03-24T12:13:10.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.7"}]: dispatch 2024-03-24T12:13:10.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.8"}]: dispatch 2024-03-24T12:13:10.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.8"}]: dispatch 2024-03-24T12:13:10.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.9"}]: dispatch 2024-03-24T12:13:10.963 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.9"}]: dispatch 2024-03-24T12:13:10.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:10.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:13:10.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-13", "mode": "writeback"}]': finished 2024-03-24T12:13:10.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: osdmap e135: 8 total, 8 up, 8 in 2024-03-24T12:13:10.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.0"}]: dispatch 2024-03-24T12:13:10.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.0"}]: dispatch 2024-03-24T12:13:10.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.1"}]: dispatch 2024-03-24T12:13:10.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.1"}]: dispatch 2024-03-24T12:13:10.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.2"}]: dispatch 2024-03-24T12:13:10.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.2"}]: dispatch 2024-03-24T12:13:10.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.3"}]: dispatch 2024-03-24T12:13:10.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.3"}]: dispatch 2024-03-24T12:13:10.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.4"}]: dispatch 2024-03-24T12:13:10.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.4"}]: dispatch 2024-03-24T12:13:10.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.5"}]: dispatch 2024-03-24T12:13:10.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.5"}]: dispatch 2024-03-24T12:13:10.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.6"}]: dispatch 2024-03-24T12:13:10.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.6"}]: dispatch 2024-03-24T12:13:10.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.7"}]: dispatch 2024-03-24T12:13:10.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.7"}]: dispatch 2024-03-24T12:13:10.964 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.8"}]: dispatch 2024-03-24T12:13:10.965 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.8"}]: dispatch 2024-03-24T12:13:10.965 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.9"}]: dispatch 2024-03-24T12:13:10.965 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:10 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.9"}]: dispatch 2024-03-24T12:13:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:13:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-13", "mode": "writeback"}]': finished 2024-03-24T12:13:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: osdmap e135: 8 total, 8 up, 8 in 2024-03-24T12:13:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.0"}]: dispatch 2024-03-24T12:13:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.0"}]: dispatch 2024-03-24T12:13:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.1"}]: dispatch 2024-03-24T12:13:11.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.1"}]: dispatch 2024-03-24T12:13:11.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.2"}]: dispatch 2024-03-24T12:13:11.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.2"}]: dispatch 2024-03-24T12:13:11.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.3"}]: dispatch 2024-03-24T12:13:11.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.3"}]: dispatch 2024-03-24T12:13:11.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.4"}]: dispatch 2024-03-24T12:13:11.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.4"}]: dispatch 2024-03-24T12:13:11.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.5"}]: dispatch 2024-03-24T12:13:11.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.5"}]: dispatch 2024-03-24T12:13:11.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.6"}]: dispatch 2024-03-24T12:13:11.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.6"}]: dispatch 2024-03-24T12:13:11.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.7"}]: dispatch 2024-03-24T12:13:11.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.7"}]: dispatch 2024-03-24T12:13:11.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.8"}]: dispatch 2024-03-24T12:13:11.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.8"}]: dispatch 2024-03-24T12:13:11.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "174.9"}]: dispatch 2024-03-24T12:13:11.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:10 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "174.9"}]: dispatch 2024-03-24T12:13:11.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:11.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[30108]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:11.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[30108]: osdmap e136: 8 total, 8 up, 8 in 2024-03-24T12:13:11.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3187927485' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi006-81775-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:11.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3747099111' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi006-81794-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:11.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi006-81794-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:11.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[30108]: 174.7 deep-scrub starts 2024-03-24T12:13:11.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[30108]: 174.7 deep-scrub ok 2024-03-24T12:13:11.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[30108]: 174.9 scrub starts 2024-03-24T12:13:11.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[30108]: 174.9 scrub ok 2024-03-24T12:13:11.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[30108]: pgmap v162: 492 pgs: 96 unknown, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 123 KiB/s wr, 120 op/s 2024-03-24T12:13:11.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[30108]: 174.6 scrub starts 2024-03-24T12:13:11.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[30108]: 174.6 scrub ok 2024-03-24T12:13:11.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[30108]: 174.4 scrub starts 2024-03-24T12:13:11.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[30108]: 174.4 scrub ok 2024-03-24T12:13:11.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:11.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[41734]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:11.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[41734]: osdmap e136: 8 total, 8 up, 8 in 2024-03-24T12:13:11.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3187927485' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi006-81775-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:11.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3747099111' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi006-81794-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:11.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi006-81794-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:11.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[41734]: 174.7 deep-scrub starts 2024-03-24T12:13:11.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[41734]: 174.7 deep-scrub ok 2024-03-24T12:13:11.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[41734]: 174.9 scrub starts 2024-03-24T12:13:11.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[41734]: 174.9 scrub ok 2024-03-24T12:13:11.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[41734]: pgmap v162: 492 pgs: 96 unknown, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 123 KiB/s wr, 120 op/s 2024-03-24T12:13:11.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[41734]: 174.6 scrub starts 2024-03-24T12:13:11.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[41734]: 174.6 scrub ok 2024-03-24T12:13:11.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[41734]: 174.4 scrub starts 2024-03-24T12:13:11.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:11 smithi006 ceph-mon[41734]: 174.4 scrub ok 2024-03-24T12:13:12.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:12.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:11 smithi090 ceph-mon[33646]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:12.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:11 smithi090 ceph-mon[33646]: osdmap e136: 8 total, 8 up, 8 in 2024-03-24T12:13:12.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3187927485' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi006-81775-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:12.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3747099111' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi006-81794-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:12.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi006-81794-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:12.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:11 smithi090 ceph-mon[33646]: 174.7 deep-scrub starts 2024-03-24T12:13:12.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:11 smithi090 ceph-mon[33646]: 174.7 deep-scrub ok 2024-03-24T12:13:12.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:11 smithi090 ceph-mon[33646]: 174.9 scrub starts 2024-03-24T12:13:12.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:11 smithi090 ceph-mon[33646]: 174.9 scrub ok 2024-03-24T12:13:12.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:11 smithi090 ceph-mon[33646]: pgmap v162: 492 pgs: 96 unknown, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 123 KiB/s wr, 120 op/s 2024-03-24T12:13:12.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:11 smithi090 ceph-mon[33646]: 174.6 scrub starts 2024-03-24T12:13:12.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:11 smithi090 ceph-mon[33646]: 174.6 scrub ok 2024-03-24T12:13:12.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:11 smithi090 ceph-mon[33646]: 174.4 scrub starts 2024-03-24T12:13:12.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:11 smithi090 ceph-mon[33646]: 174.4 scrub ok 2024-03-24T12:13:12.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:13:12 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:13:12.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[30108]: 174.3 scrub starts 2024-03-24T12:13:12.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[30108]: 174.3 scrub ok 2024-03-24T12:13:12.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[30108]: 174.0 scrub starts 2024-03-24T12:13:12.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[30108]: 174.0 scrub ok 2024-03-24T12:13:12.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[30108]: 174.8 scrub starts 2024-03-24T12:13:12.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[30108]: 174.8 scrub ok 2024-03-24T12:13:12.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:12.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3187927485' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi006-81775-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:12.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi006-81794-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:12.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[30108]: osdmap e137: 8 total, 8 up, 8 in 2024-03-24T12:13:12.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:13:12.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[30108]: 174.2 scrub starts 2024-03-24T12:13:12.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[30108]: 174.2 scrub ok 2024-03-24T12:13:12.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[41734]: 174.3 scrub starts 2024-03-24T12:13:12.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[41734]: 174.3 scrub ok 2024-03-24T12:13:12.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[41734]: 174.0 scrub starts 2024-03-24T12:13:12.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[41734]: 174.0 scrub ok 2024-03-24T12:13:12.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[41734]: 174.8 scrub starts 2024-03-24T12:13:12.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[41734]: 174.8 scrub ok 2024-03-24T12:13:12.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:12.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3187927485' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi006-81775-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:12.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi006-81794-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:12.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[41734]: osdmap e137: 8 total, 8 up, 8 in 2024-03-24T12:13:12.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:13:12.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[41734]: 174.2 scrub starts 2024-03-24T12:13:12.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:12 smithi006 ceph-mon[41734]: 174.2 scrub ok 2024-03-24T12:13:13.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:12 smithi090 ceph-mon[33646]: 174.3 scrub starts 2024-03-24T12:13:13.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:12 smithi090 ceph-mon[33646]: 174.3 scrub ok 2024-03-24T12:13:13.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:12 smithi090 ceph-mon[33646]: 174.0 scrub starts 2024-03-24T12:13:13.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:12 smithi090 ceph-mon[33646]: 174.0 scrub ok 2024-03-24T12:13:13.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:12 smithi090 ceph-mon[33646]: 174.8 scrub starts 2024-03-24T12:13:13.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:12 smithi090 ceph-mon[33646]: 174.8 scrub ok 2024-03-24T12:13:13.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:12 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:13.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:12 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3187927485' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi006-81775-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:13.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi006-81794-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:13.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:12 smithi090 ceph-mon[33646]: osdmap e137: 8 total, 8 up, 8 in 2024-03-24T12:13:13.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:12 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:13:13.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:12 smithi090 ceph-mon[33646]: 174.2 scrub starts 2024-03-24T12:13:13.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:12 smithi090 ceph-mon[33646]: 174.2 scrub ok 2024-03-24T12:13:13.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:13 smithi006 ceph-mon[30108]: 174.1 scrub starts 2024-03-24T12:13:13.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:13 smithi006 ceph-mon[30108]: 174.1 scrub ok 2024-03-24T12:13:13.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:13 smithi006 ceph-mon[30108]: 174.5 scrub starts 2024-03-24T12:13:13.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:13 smithi006 ceph-mon[30108]: 174.5 scrub ok 2024-03-24T12:13:13.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:13 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:13.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:13 smithi006 ceph-mon[30108]: osdmap e138: 8 total, 8 up, 8 in 2024-03-24T12:13:13.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:13 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2037860533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:13.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:13 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:13.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:13 smithi006 ceph-mon[30108]: pgmap v165: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 55 KiB/s rd, 76 op/s 2024-03-24T12:13:13.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:13 smithi006 ceph-mon[41734]: 174.1 scrub starts 2024-03-24T12:13:13.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:13 smithi006 ceph-mon[41734]: 174.1 scrub ok 2024-03-24T12:13:13.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:13 smithi006 ceph-mon[41734]: 174.5 scrub starts 2024-03-24T12:13:13.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:13 smithi006 ceph-mon[41734]: 174.5 scrub ok 2024-03-24T12:13:13.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:13 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:13.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:13 smithi006 ceph-mon[41734]: osdmap e138: 8 total, 8 up, 8 in 2024-03-24T12:13:13.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:13 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2037860533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:13.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:13 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:13.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:13 smithi006 ceph-mon[41734]: pgmap v165: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 55 KiB/s rd, 76 op/s 2024-03-24T12:13:14.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:13 smithi090 ceph-mon[33646]: 174.1 scrub starts 2024-03-24T12:13:14.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:13 smithi090 ceph-mon[33646]: 174.1 scrub ok 2024-03-24T12:13:14.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:13 smithi090 ceph-mon[33646]: 174.5 scrub starts 2024-03-24T12:13:14.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:13 smithi090 ceph-mon[33646]: 174.5 scrub ok 2024-03-24T12:13:14.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:13 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:14.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:13 smithi090 ceph-mon[33646]: osdmap e138: 8 total, 8 up, 8 in 2024-03-24T12:13:14.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:13 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2037860533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:14.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:13 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:14.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:13 smithi090 ceph-mon[33646]: pgmap v165: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 55 KiB/s rd, 76 op/s 2024-03-24T12:13:14.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started api_snapshots: Running main() from gmock_main.cc 2024-03-24T12:13:14.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2024-03-24T12:13:14.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [----------] Global test environment set-up. 2024-03-24T12:13:14.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-03-24T12:13:14.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-03-24T12:13:14.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (3163 ms) 2024-03-24T12:13:14.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-03-24T12:13:14.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (3177 ms) 2024-03-24T12:13:14.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-03-24T12:13:14.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (3139 ms) 2024-03-24T12:13:14.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-03-24T12:13:14.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (2689 ms) 2024-03-24T12:13:14.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (12169 ms total) 2024-03-24T12:13:14.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: 2024-03-24T12:13:14.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-03-24T12:13:14.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-03-24T12:13:14.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (7130 ms) 2024-03-24T12:13:14.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-03-24T12:13:14.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (6075 ms) 2024-03-24T12:13:14.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-03-24T12:13:14.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (4970 ms) 2024-03-24T12:13:14.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (18175 ms total) 2024-03-24T12:13:14.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: 2024-03-24T12:13:14.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-03-24T12:13:14.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-03-24T12:13:14.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (3016 ms) 2024-03-24T12:13:14.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-03-24T12:13:14.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (2002 ms) 2024-03-24T12:13:14.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-03-24T12:13:14.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (1973 ms) 2024-03-24T12:13:14.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-03-24T12:13:14.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (2011 ms) 2024-03-24T12:13:14.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (9002 ms total) 2024-03-24T12:13:14.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: 2024-03-24T12:13:14.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-03-24T12:13:14.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-03-24T12:13:14.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (4033 ms) 2024-03-24T12:13:14.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-03-24T12:13:14.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (4008 ms) 2024-03-24T12:13:14.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8041 ms total) 2024-03-24T12:13:14.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: 2024-03-24T12:13:14.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [----------] Global test environment tear-down 2024-03-24T12:13:14.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (73366 ms total) 2024-03-24T12:13:14.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots: [ PASSED ] 13 tests. 2024-03-24T12:13:14.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:14.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:14 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]': finished 2024-03-24T12:13:14.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:14 smithi006 ceph-mon[30108]: osdmap e139: 8 total, 8 up, 8 in 2024-03-24T12:13:14.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2037860533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:14.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2830998409' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi006-81775-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:14.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:14 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:14.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2211801852' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi006-81794-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:14.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:14.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:14 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]': finished 2024-03-24T12:13:14.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:14 smithi006 ceph-mon[41734]: osdmap e139: 8 total, 8 up, 8 in 2024-03-24T12:13:14.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2037860533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:14.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2830998409' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi006-81775-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:14.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:14 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:14.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2211801852' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi006-81794-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:14 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]': finished 2024-03-24T12:13:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:14 smithi090 ceph-mon[33646]: osdmap e139: 8 total, 8 up, 8 in 2024-03-24T12:13:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2037860533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2830998409' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi006-81775-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:15.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:14 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]: dispatch 2024-03-24T12:13:15.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2211801852' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi006-81794-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:15.687 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:13:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:13:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:13:15.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:15.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2830998409' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi006-81775-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:15.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:15 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]': finished 2024-03-24T12:13:15.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2211801852' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi006-81794-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:15.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:15 smithi006 ceph-mon[30108]: osdmap e140: 8 total, 8 up, 8 in 2024-03-24T12:13:15.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-24T12:13:15.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:15.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:15 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:15.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:15 smithi006 ceph-mon[30108]: pgmap v168: 484 pgs: 96 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 55 KiB/s rd, 76 op/s 2024-03-24T12:13:15.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:15.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2830998409' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi006-81775-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:15.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:15 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]': finished 2024-03-24T12:13:15.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2211801852' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi006-81794-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:15.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:15 smithi006 ceph-mon[41734]: osdmap e140: 8 total, 8 up, 8 in 2024-03-24T12:13:15.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-24T12:13:15.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:15.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:15 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:15.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:15 smithi006 ceph-mon[41734]: pgmap v168: 484 pgs: 96 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 55 KiB/s rd, 76 op/s 2024-03-24T12:13:16.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:16.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2830998409' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi006-81775-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:16.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:15 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi006-82076-15"}]': finished 2024-03-24T12:13:16.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2211801852' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi006-81794-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:16.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:15 smithi090 ceph-mon[33646]: osdmap e140: 8 total, 8 up, 8 in 2024-03-24T12:13:16.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-24T12:13:16.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:15 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:15 smithi090 ceph-mon[33646]: pgmap v168: 484 pgs: 96 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 55 KiB/s rd, 76 op/s 2024-03-24T12:13:16.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:16 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:16.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:16 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:16.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:16 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:16.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:16 smithi006 ceph-mon[30108]: osdmap e141: 8 total, 8 up, 8 in 2024-03-24T12:13:16.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:16 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "app2"}]: dispatch 2024-03-24T12:13:16.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:16 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:16.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:16 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:16.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:16 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:16.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:16 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:16.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:16 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:16.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:16 smithi006 ceph-mon[41734]: osdmap e141: 8 total, 8 up, 8 in 2024-03-24T12:13:16.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:16 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "app2"}]: dispatch 2024-03-24T12:13:16.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:16 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:16.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:16 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:17.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:16 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:17.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:16 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:17.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:16 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:17.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:16 smithi090 ceph-mon[33646]: osdmap e141: 8 total, 8 up, 8 in 2024-03-24T12:13:17.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:16 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "app2"}]: dispatch 2024-03-24T12:13:17.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:16 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:17.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:16 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:17.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:17.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:17.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[30108]: osdmap e142: 8 total, 8 up, 8 in 2024-03-24T12:13:17.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2047873048' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi006-81794-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:17.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi006-82056-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-03-24T12:13:17.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/424075752' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi006-81775-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:17.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-24T12:13:17.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi006-81775-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:17.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-24T12:13:17.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[30108]: pgmap v171: 452 pgs: 16 creating+peering, 70 unknown, 366 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-24T12:13:17.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:13:17.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:13:17.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:17.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:17.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[41734]: osdmap e142: 8 total, 8 up, 8 in 2024-03-24T12:13:17.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2047873048' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi006-81794-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:17.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi006-82056-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-03-24T12:13:17.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/424075752' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi006-81775-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:17.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-24T12:13:17.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi006-81775-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:17.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-24T12:13:17.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[41734]: pgmap v171: 452 pgs: 16 creating+peering, 70 unknown, 366 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-24T12:13:17.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:13:17.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:17 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:13:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi006-82056-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:17 smithi090 ceph-mon[33646]: osdmap e142: 8 total, 8 up, 8 in 2024-03-24T12:13:18.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2047873048' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi006-81794-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:18.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi006-82056-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-03-24T12:13:18.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/424075752' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi006-81775-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:18.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-24T12:13:18.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi006-81775-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:18.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-24T12:13:18.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:17 smithi090 ceph-mon[33646]: pgmap v171: 452 pgs: 16 creating+peering, 70 unknown, 366 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-24T12:13:18.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:17 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:13:18.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:17 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:13:18.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:18.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2047873048' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi006-81794-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:18.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi006-81775-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:18.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-24T12:13:18.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:18 smithi006 ceph-mon[30108]: osdmap e143: 8 total, 8 up, 8 in 2024-03-24T12:13:18.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-24T12:13:18.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-24T12:13:18.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/441067820' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82137-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:18.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:18.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2047873048' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi006-81794-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:18.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi006-81775-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:18.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-24T12:13:18.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:18 smithi006 ceph-mon[41734]: osdmap e143: 8 total, 8 up, 8 in 2024-03-24T12:13:18.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-24T12:13:18.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-24T12:13:18.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/441067820' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82137-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:19.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:19.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2047873048' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi006-81794-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:19.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi006-81775-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:19.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-24T12:13:19.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:18 smithi090 ceph-mon[33646]: osdmap e143: 8 total, 8 up, 8 in 2024-03-24T12:13:19.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-24T12:13:19.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-24T12:13:19.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/441067820' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82137-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:19.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:19.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-24T12:13:19.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/441067820' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82137-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:19.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:19 smithi006 ceph-mon[30108]: osdmap e144: 8 total, 8 up, 8 in 2024-03-24T12:13:19.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key1"}]: dispatch 2024-03-24T12:13:19.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key1"}]: dispatch 2024-03-24T12:13:19.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:19 smithi006 ceph-mon[30108]: pgmap v174: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-24T12:13:19.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:19.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-24T12:13:19.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/441067820' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82137-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:19.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:19 smithi006 ceph-mon[41734]: osdmap e144: 8 total, 8 up, 8 in 2024-03-24T12:13:19.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key1"}]: dispatch 2024-03-24T12:13:19.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key1"}]: dispatch 2024-03-24T12:13:19.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:19 smithi006 ceph-mon[41734]: pgmap v174: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-24T12:13:20.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:20.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-24T12:13:20.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/441067820' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82137-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:20.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:19 smithi090 ceph-mon[33646]: osdmap e144: 8 total, 8 up, 8 in 2024-03-24T12:13:20.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3579781590' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key1"}]: dispatch 2024-03-24T12:13:20.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key1"}]: dispatch 2024-03-24T12:13:20.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:19 smithi090 ceph-mon[33646]: pgmap v174: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-24T12:13:20.817 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-03-24T12:13:20.817 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-03-24T12:13:20.817 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-03-24T12:13:20.817 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-03-24T12:13:20.817 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-03-24T12:13:20.817 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-03-24T12:13:20.817 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: 2024-03-24T12:13:20.817 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-03-24T12:13:20.817 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: seed 82056 2024-03-24T12:13:20.817 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-03-24T12:13:20.818 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (0 ms) 2024-03-24T12:13:20.818 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-03-24T12:13:20.818 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (345 ms) 2024-03-24T12:13:20.818 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-03-24T12:13:20.818 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (16 ms) 2024-03-24T12:13:20.818 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-03-24T12:13:20.818 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (9 ms) 2024-03-24T12:13:20.818 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-03-24T12:13:20.818 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (10 ms) 2024-03-24T12:13:20.818 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-03-24T12:13:20.818 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (2 ms) 2024-03-24T12:13:20.818 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-03-24T12:13:20.818 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (2 ms) 2024-03-24T12:13:20.818 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-03-24T12:13:20.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (7 ms) 2024-03-24T12:13:20.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-03-24T12:13:20.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (3 ms) 2024-03-24T12:13:20.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-03-24T12:13:20.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (7 ms) 2024-03-24T12:13:20.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-03-24T12:13:20.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (3 ms) 2024-03-24T12:13:20.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-03-24T12:13:20.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (5 ms) 2024-03-24T12:13:20.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-03-24T12:13:20.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (8 ms) 2024-03-24T12:13:20.819 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-03-24T12:13:20.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: osd_max_attr_size = 0 2024-03-24T12:13:20.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-03-24T12:13:20.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (1649 ms) 2024-03-24T12:13:20.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-03-24T12:13:20.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (454 ms) 2024-03-24T12:13:20.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-03-24T12:13:20.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-03-24T12:13:20.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: done waiting, doing copies 2024-03-24T12:13:20.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: waiting for final deep scrubs... 2024-03-24T12:13:20.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: done waiting 2024-03-24T12:13:20.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60680 ms) 2024-03-24T12:13:20.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-03-24T12:13:20.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (5 ms) 2024-03-24T12:13:20.820 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-03-24T12:13:20.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (6 ms) 2024-03-24T12:13:20.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-03-24T12:13:20.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4820 ms) 2024-03-24T12:13:20.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-03-24T12:13:20.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (1 ms) 2024-03-24T12:13:20.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-03-24T12:13:20.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-03-24T12:13:20.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-03-24T12:13:20.821 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-03-24T12:13:20.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:20 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:20.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:20 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key1"}]': finished 2024-03-24T12:13:20.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:20 smithi006 ceph-mon[30108]: osdmap e145: 8 total, 8 up, 8 in 2024-03-24T12:13:20.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:20 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/717955506' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi006-81794-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:20.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:20 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi006-81794-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:20.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:20 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2964459127' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi006-81775-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:20.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:20 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi006-81775-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:20.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:20 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:20.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:20 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key1"}]': finished 2024-03-24T12:13:20.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:20 smithi006 ceph-mon[41734]: osdmap e145: 8 total, 8 up, 8 in 2024-03-24T12:13:20.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:20 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/717955506' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi006-81794-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:20.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:20 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi006-81794-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:20.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:20 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2964459127' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi006-81775-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:20.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:20 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi006-81775-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:21.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:20 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:21.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:20 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi006-82056-1","app":"app1","key":"key1"}]': finished 2024-03-24T12:13:21.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:20 smithi090 ceph-mon[33646]: osdmap e145: 8 total, 8 up, 8 in 2024-03-24T12:13:21.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:20 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/717955506' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi006-81794-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:21.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:20 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi006-81794-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:21.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:20 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2964459127' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi006-81775-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:21.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:20 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi006-81775-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:21.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:21.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:21 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi006-81794-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:21.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:21 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi006-81775-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:21.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:21 smithi006 ceph-mon[30108]: osdmap e146: 8 total, 8 up, 8 in 2024-03-24T12:13:21.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:21.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:21 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:21.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:21.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:21 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:21.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82056-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:21.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:21 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82056-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:21.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:21 smithi006 ceph-mon[30108]: pgmap v177: 452 pgs: 96 unknown, 356 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-24T12:13:21.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:21.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:21 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi006-81794-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:21.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:21 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi006-81775-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:21.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:21 smithi006 ceph-mon[41734]: osdmap e146: 8 total, 8 up, 8 in 2024-03-24T12:13:21.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:21.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:21 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:21.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:21.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:21 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:21.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82056-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:21.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:21 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82056-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:21.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:21 smithi006 ceph-mon[41734]: pgmap v177: 452 pgs: 96 unknown, 356 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-24T12:13:22.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:22.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:21 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi006-81794-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:22.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:21 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi006-81775-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:22.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:21 smithi090 ceph-mon[33646]: osdmap e146: 8 total, 8 up, 8 in 2024-03-24T12:13:22.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:22.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:21 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:22.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:22.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:21 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:22.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82056-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:22.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:21 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82056-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:22.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:21 smithi090 ceph-mon[33646]: pgmap v177: 452 pgs: 96 unknown, 356 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-24T12:13:22.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:13:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:13:22.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:22.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82056-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:22.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[30108]: osdmap e147: 8 total, 8 up, 8 in 2024-03-24T12:13:22.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82056-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:22.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82056-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:22.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2470384770' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:22.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:13:22.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:22.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2470384770' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:22.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:22.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2470384770' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:22.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:22.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:22.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82056-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:22.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[41734]: osdmap e147: 8 total, 8 up, 8 in 2024-03-24T12:13:22.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82056-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:22.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82056-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:22.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2470384770' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:22.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:13:22.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:22.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2470384770' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:22.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:22.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2470384770' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:22.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:23.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:23.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82056-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:23.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:22 smithi090 ceph-mon[33646]: osdmap e147: 8 total, 8 up, 8 in 2024-03-24T12:13:23.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82056-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:23.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82056-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:23.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2470384770' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:23.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:22 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:13:23.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:23.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2470384770' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:23.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:23.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2470384770' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:23.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:23.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:23.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:23.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2470384770' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi006-82137-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:23.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:23 smithi006 ceph-mon[30108]: pgmap v180: 420 pgs: 64 unknown, 356 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:13:23.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:23 smithi006 ceph-mon[30108]: osdmap e148: 8 total, 8 up, 8 in 2024-03-24T12:13:23.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi006-82137-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:23.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2777671873' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi006-81775-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:23.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4242598798' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi006-81794-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:23.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi006-81775-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:23.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi006-81794-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:23.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:23.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:23.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2470384770' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi006-82137-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:23.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:23 smithi006 ceph-mon[41734]: pgmap v180: 420 pgs: 64 unknown, 356 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:13:23.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:23 smithi006 ceph-mon[41734]: osdmap e148: 8 total, 8 up, 8 in 2024-03-24T12:13:23.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi006-82137-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:23.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2777671873' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi006-81775-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:23.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4242598798' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi006-81794-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:23.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi006-81775-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:23.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi006-81794-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:24.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:24.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:24.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2470384770' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi006-82137-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:24.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:23 smithi090 ceph-mon[33646]: pgmap v180: 420 pgs: 64 unknown, 356 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:13:24.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:23 smithi090 ceph-mon[33646]: osdmap e148: 8 total, 8 up, 8 in 2024-03-24T12:13:24.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi006-82137-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:24.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2777671873' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi006-81775-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:24.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4242598798' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi006-81794-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:24.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi006-81775-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:24.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi006-81794-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:24.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:24.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:24 smithi006 ceph-mon[30108]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:24.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82056-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82056-24"}]': finished 2024-03-24T12:13:24.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi006-81775-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:24.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi006-81794-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:24.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:24 smithi006 ceph-mon[30108]: osdmap e149: 8 total, 8 up, 8 in 2024-03-24T12:13:24.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:24.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:24 smithi006 ceph-mon[41734]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:24.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82056-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82056-24"}]': finished 2024-03-24T12:13:24.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi006-81775-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:24.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi006-81794-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:24.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:24 smithi006 ceph-mon[41734]: osdmap e149: 8 total, 8 up, 8 in 2024-03-24T12:13:25.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:25.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:24 smithi090 ceph-mon[33646]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:25.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82056-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82056-24"}]': finished 2024-03-24T12:13:25.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi006-81775-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:25.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi006-81794-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:25.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:24 smithi090 ceph-mon[33646]: osdmap e149: 8 total, 8 up, 8 in 2024-03-24T12:13:25.705 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:13:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:13:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:13:25.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:25.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:25 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi006-82137-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]': finished 2024-03-24T12:13:25.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:25 smithi006 ceph-mon[30108]: osdmap e150: 8 total, 8 up, 8 in 2024-03-24T12:13:25.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82056-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:25.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:25 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82056-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:25.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:25 smithi006 ceph-mon[30108]: pgmap v183: 404 pgs: 48 unknown, 356 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:13:25.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:25.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:25 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi006-82137-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]': finished 2024-03-24T12:13:25.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:25 smithi006 ceph-mon[41734]: osdmap e150: 8 total, 8 up, 8 in 2024-03-24T12:13:25.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82056-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:25.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:25 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82056-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:25.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:25 smithi006 ceph-mon[41734]: pgmap v183: 404 pgs: 48 unknown, 356 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:13:26.079 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:26.079 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:25 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi006-82137-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]': finished 2024-03-24T12:13:26.080 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:25 smithi090 ceph-mon[33646]: osdmap e150: 8 total, 8 up, 8 in 2024-03-24T12:13:26.080 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82056-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:26.080 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:25 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82056-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:26.080 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:25 smithi090 ceph-mon[33646]: pgmap v183: 404 pgs: 48 unknown, 356 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:13:26.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:26.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82056-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:26.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:26 smithi006 ceph-mon[30108]: osdmap e151: 8 total, 8 up, 8 in 2024-03-24T12:13:26.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82056-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:26.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2563880393' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi006-81794-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:26.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82056-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:26.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi006-81794-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:26.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1633342435' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi006-81775-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:26.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:26.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82056-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:26.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:26 smithi006 ceph-mon[41734]: osdmap e151: 8 total, 8 up, 8 in 2024-03-24T12:13:26.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82056-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:26.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2563880393' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi006-81794-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:26.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82056-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:26.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi006-81794-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:26.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1633342435' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi006-81775-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:27.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:27.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82056-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:27.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:26 smithi090 ceph-mon[33646]: osdmap e151: 8 total, 8 up, 8 in 2024-03-24T12:13:27.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82056-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:27.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2563880393' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi006-81794-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:27.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82056-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:27.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi006-81794-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:27.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1633342435' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi006-81775-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:27.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:27 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:27.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:27 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82056-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:27.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:27 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi006-81794-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:27.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:27 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1633342435' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi006-81775-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:27.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:27 smithi006 ceph-mon[30108]: osdmap e152: 8 total, 8 up, 8 in 2024-03-24T12:13:27.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:27 smithi006 ceph-mon[30108]: pgmap v186: 468 pgs: 91 unknown, 377 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 255 B/s wr, 0 op/s 2024-03-24T12:13:27.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:27 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:27.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:27 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82056-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:27.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:27 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi006-81794-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:27.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:27 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1633342435' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi006-81775-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:27.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:27 smithi006 ceph-mon[41734]: osdmap e152: 8 total, 8 up, 8 in 2024-03-24T12:13:27.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:27 smithi006 ceph-mon[41734]: pgmap v186: 468 pgs: 91 unknown, 377 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 255 B/s wr, 0 op/s 2024-03-24T12:13:28.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:27 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:28.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:27 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82056-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:28.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:27 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi006-81794-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:28.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:27 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1633342435' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi006-81775-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:28.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:27 smithi090 ceph-mon[33646]: osdmap e152: 8 total, 8 up, 8 in 2024-03-24T12:13:28.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:27 smithi090 ceph-mon[33646]: pgmap v186: 468 pgs: 91 unknown, 377 active+clean; 592 KiB data, 605 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 255 B/s wr, 0 op/s 2024-03-24T12:13:28.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:28.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:28 smithi006 ceph-mon[30108]: osdmap e153: 8 total, 8 up, 8 in 2024-03-24T12:13:28.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:28.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:28 smithi006 ceph-mon[41734]: osdmap e153: 8 total, 8 up, 8 in 2024-03-24T12:13:29.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:29.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:28 smithi090 ceph-mon[33646]: osdmap e153: 8 total, 8 up, 8 in 2024-03-24T12:13:29.712 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:29 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:29.712 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:29 smithi090 ceph-mon[33646]: osdmap e154: 8 total, 8 up, 8 in 2024-03-24T12:13:29.712 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:29 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/345981540' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi006-81794-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:29.713 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:29 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:29.713 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi006-81794-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:29.713 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:29.859 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP ( api_snapshots_pp: Running main() from gmock_main.cc 2024-03-24T12:13:29.859 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [==========] Running 21 tests from 5 test suites. 2024-03-24T12:13:29.860 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-03-24T12:13:29.860 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-03-24T12:13:29.860 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: seed 82137 2024-03-24T12:13:29.860 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-03-24T12:13:29.860 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (6380 ms) 2024-03-24T12:13:29.860 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-03-24T12:13:29.860 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (4824 ms) 2024-03-24T12:13:29.860 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-03-24T12:13:29.860 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (2193 ms) 2024-03-24T12:13:29.860 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-03-24T12:13:29.860 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (3678 ms) 2024-03-24T12:13:29.860 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-03-24T12:13:29.860 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (6902 ms) 2024-03-24T12:13:29.861 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (23977 ms total) 2024-03-24T12:13:29.861 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: 2024-03-24T12:13:29.861 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-03-24T12:13:29.861 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-03-24T12:13:29.861 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (3919 ms) 2024-03-24T12:13:29.861 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-03-24T12:13:29.861 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (3928 ms) 2024-03-24T12:13:29.861 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-03-24T12:13:29.861 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (6010 ms) 2024-03-24T12:13:29.861 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-03-24T12:13:29.861 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (3986 ms) 2024-03-24T12:13:29.861 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-03-24T12:13:29.861 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2058 ms) 2024-03-24T12:13:29.861 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-03-24T12:13:29.861 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1667-gdb0330b1/rpm/el9/BUILD/ceph-19.0.0-1667-gdb0330b1/src/test/librados/snapshots_cxx.cc:463: Skipped 2024-03-24T12:13:29.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: 2024-03-24T12:13:29.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-03-24T12:13:29.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-03-24T12:13:29.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: deleting snap 14 in pool LibRadosSnapshotsSelfManagedPP_smithi006-82137-7 2024-03-24T12:13:29.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: waiting for snaps to purge 2024-03-24T12:13:29.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17954 ms) 2024-03-24T12:13:29.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (37855 ms total) 2024-03-24T12:13:29.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: 2024-03-24T12:13:29.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-03-24T12:13:29.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-03-24T12:13:29.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (9 ms) 2024-03-24T12:13:29.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-03-24T12:13:29.862 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (4997 ms) 2024-03-24T12:13:29.863 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5006 ms total) 2024-03-24T12:13:29.863 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: 2024-03-24T12:13:29.863 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-03-24T12:13:29.863 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-03-24T12:13:29.863 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (3022 ms) 2024-03-24T12:13:29.863 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-03-24T12:13:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:29 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:29 smithi006 ceph-mon[30108]: osdmap e154: 8 total, 8 up, 8 in 2024-03-24T12:13:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:29 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/345981540' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi006-81794-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:29 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi006-81794-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:29 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2211967207' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi006-81775-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi006-81775-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:29 smithi006 ceph-mon[30108]: pgmap v189: 428 pgs: 64 unknown, 364 active+clean; 592 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:13:29.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:29 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:29.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:29 smithi006 ceph-mon[41734]: osdmap e154: 8 total, 8 up, 8 in 2024-03-24T12:13:29.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:29 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/345981540' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi006-81794-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:29.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:29 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:29.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi006-81794-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:29.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:29.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:29 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2211967207' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi006-81775-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:29.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi006-81775-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:29.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:29 smithi006 ceph-mon[41734]: pgmap v189: 428 pgs: 64 unknown, 364 active+clean; 592 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:13:30.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:29 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2211967207' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi006-81775-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:30.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi006-81775-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:30.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:29 smithi090 ceph-mon[33646]: pgmap v189: 428 pgs: 64 unknown, 364 active+clean; 592 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:13:30.873 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ OK ] LibRadosS snapshots: Running main() from gmock_main.cc 2024-03-24T12:13:30.873 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [==========] Running 11 tests from 2 test suites. 2024-03-24T12:13:30.873 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [----------] Global test environment set-up. 2024-03-24T12:13:30.873 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots 2024-03-24T12:13:30.873 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapList 2024-03-24T12:13:30.873 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapList (18539 ms) 2024-03-24T12:13:30.873 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapRemove 2024-03-24T12:13:30.873 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapRemove (5421 ms) 2024-03-24T12:13:30.873 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [ RUN ] NeoRadosSnapshots.Rollback 2024-03-24T12:13:30.873 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [ OK ] NeoRadosSnapshots.Rollback (5009 ms) 2024-03-24T12:13:30.873 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapGetName 2024-03-24T12:13:30.874 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapGetName (5036 ms) 2024-03-24T12:13:30.874 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-03-24T12:13:30.874 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapCreateRemove (7009 ms) 2024-03-24T12:13:30.874 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots (41014 ms total) 2024-03-24T12:13:30.874 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: 2024-03-24T12:13:30.874 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps 2024-03-24T12:13:30.874 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-03-24T12:13:30.874 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Snap (5278 ms) 2024-03-24T12:13:30.874 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-03-24T12:13:30.874 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Rollback (5752 ms) 2024-03-24T12:13:30.874 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-03-24T12:13:30.874 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (7968 ms) 2024-03-24T12:13:30.874 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-03-24T12:13:30.874 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Bug11677 (6010 ms) 2024-03-24T12:13:30.874 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-03-24T12:13:30.875 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.OrderSnap (3994 ms) 2024-03-24T12:13:30.875 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-03-24T12:13:30.875 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: Deleting snap 3 in pool ReusePurgedSnapsmithi006-83060-11. 2024-03-24T12:13:30.875 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: Waiting for snaps to purge. 2024-03-24T12:13:30.875 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (19051 ms) 2024-03-24T12:13:30.875 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps (48054 ms total) 2024-03-24T12:13:30.875 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: 2024-03-24T12:13:30.875 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [----------] Global test environment tear-down 2024-03-24T12:13:30.875 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [==========] 11 tests from 2 test suites ran. (89069 ms total) 2024-03-24T12:13:30.875 INFO:tasks.workunit.client.0.smithi006.stdout: snapshots: [ PASSED ] 11 tests. 2024-03-24T12:13:31.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:30 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:31.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:30 smithi090 ceph-mon[33646]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:31.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:30 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi006-81794-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:31.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:30 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82056-24"}]': finished 2024-03-24T12:13:31.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:30 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi006-81775-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:31.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:30 smithi090 ceph-mon[33646]: osdmap e155: 8 total, 8 up, 8 in 2024-03-24T12:13:31.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:30 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:31.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:30 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:31.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:30 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:31.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:30 smithi006 ceph-mon[30108]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:31.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:30 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi006-81794-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:31.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:30 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82056-24"}]': finished 2024-03-24T12:13:31.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:30 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi006-81775-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:31.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:30 smithi006 ceph-mon[30108]: osdmap e155: 8 total, 8 up, 8 in 2024-03-24T12:13:31.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:30 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:31.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:30 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:31.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:30 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:31.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:30 smithi006 ceph-mon[41734]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:31.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:30 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi006-81794-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:31.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:30 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82056-24"}]': finished 2024-03-24T12:13:31.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:30 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi006-81775-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:31.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:30 smithi006 ceph-mon[41734]: osdmap e155: 8 total, 8 up, 8 in 2024-03-24T12:13:31.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:30 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3534679120' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:31.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:30 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82056-24"}]: dispatch 2024-03-24T12:13:32.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:32.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:31 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82056-24"}]': finished 2024-03-24T12:13:32.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:31 smithi090 ceph-mon[33646]: osdmap e156: 8 total, 8 up, 8 in 2024-03-24T12:13:32.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:31 smithi090 ceph-mon[33646]: pgmap v192: 332 pgs: 332 active+clean; 592 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-24T12:13:32.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:31 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82056-24"}]': finished 2024-03-24T12:13:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:31 smithi006 ceph-mon[30108]: osdmap e156: 8 total, 8 up, 8 in 2024-03-24T12:13:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:31 smithi006 ceph-mon[30108]: pgmap v192: 332 pgs: 332 active+clean; 592 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-24T12:13:32.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:32.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:31 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82056-24"}]': finished 2024-03-24T12:13:32.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:31 smithi006 ceph-mon[41734]: osdmap e156: 8 total, 8 up, 8 in 2024-03-24T12:13:32.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:31 smithi006 ceph-mon[41734]: pgmap v192: 332 pgs: 332 active+clean; 592 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-24T12:13:32.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:13:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:13:33.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:32 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:33.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:32 smithi090 ceph-mon[33646]: osdmap e157: 8 total, 8 up, 8 in 2024-03-24T12:13:33.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:32 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/180351085' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi006-81794-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:33.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:32 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/636566319' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi006-81775-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:33.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:32 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1184652704' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi006-82056-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:33.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:32 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi006-81775-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:33.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:32 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi006-82056-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:33.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:32 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:13:33.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:32 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:13:33.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:32 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:33.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:32 smithi006 ceph-mon[30108]: osdmap e157: 8 total, 8 up, 8 in 2024-03-24T12:13:33.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:32 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/180351085' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi006-81794-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:33.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:32 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/636566319' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi006-81775-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:33.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:32 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1184652704' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi006-82056-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:33.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:32 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi006-81775-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:33.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:32 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi006-82056-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:33.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:32 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:13:33.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:32 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:13:33.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:32 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:33.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:32 smithi006 ceph-mon[41734]: osdmap e157: 8 total, 8 up, 8 in 2024-03-24T12:13:33.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:32 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/180351085' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi006-81794-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:33.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:32 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/636566319' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi006-81775-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:33.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:32 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1184652704' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi006-82056-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:33.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:32 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi006-81775-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:33.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:32 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi006-82056-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:33.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:32 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:13:33.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:32 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:13:34.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:34.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pg_num","val":"11"}]: dispatch 2024-03-24T12:13:34.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:33 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pg_num","val":"11"}]: dispatch 2024-03-24T12:13:34.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/180351085' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi006-81794-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:34.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:33 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi006-81775-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:34.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:33 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi006-82056-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:34.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:33 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pg_num","val":"11"}]': finished 2024-03-24T12:13:34.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:33 smithi090 ceph-mon[33646]: osdmap e158: 8 total, 8 up, 8 in 2024-03-24T12:13:34.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:34.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:34.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-24T12:13:34.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:33 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-24T12:13:34.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:33 smithi090 ceph-mon[33646]: pgmap v195: 428 pgs: 96 creating+peering, 332 active+clean; 592 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-24T12:13:34.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:34.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pg_num","val":"11"}]: dispatch 2024-03-24T12:13:34.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pg_num","val":"11"}]: dispatch 2024-03-24T12:13:34.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/180351085' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi006-81794-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:34.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi006-81775-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:34.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi006-82056-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:34.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pg_num","val":"11"}]': finished 2024-03-24T12:13:34.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[41734]: osdmap e158: 8 total, 8 up, 8 in 2024-03-24T12:13:34.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:34.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:34.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-24T12:13:34.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-24T12:13:34.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[41734]: pgmap v195: 428 pgs: 96 creating+peering, 332 active+clean; 592 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-24T12:13:34.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:34.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pg_num","val":"11"}]: dispatch 2024-03-24T12:13:34.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pg_num","val":"11"}]: dispatch 2024-03-24T12:13:34.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/180351085' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi006-81794-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:34.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi006-81775-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:34.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi006-82056-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:34.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pg_num","val":"11"}]': finished 2024-03-24T12:13:34.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[30108]: osdmap e158: 8 total, 8 up, 8 in 2024-03-24T12:13:34.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:34.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:34.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-24T12:13:34.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-24T12:13:34.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:33 smithi006 ceph-mon[30108]: pgmap v195: 428 pgs: 96 creating+peering, 332 active+clean; 592 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-24T12:13:35.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:34 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pgp_num","val":"11"}]': finished 2024-03-24T12:13:35.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:35.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:34 smithi090 ceph-mon[33646]: osdmap e159: 8 total, 8 up, 8 in 2024-03-24T12:13:35.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:34 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pgp_num","val":"11"}]': finished 2024-03-24T12:13:35.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:35.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:34 smithi006 ceph-mon[30108]: osdmap e159: 8 total, 8 up, 8 in 2024-03-24T12:13:35.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:34 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pgp_num","val":"11"}]': finished 2024-03-24T12:13:35.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:35.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:34 smithi006 ceph-mon[41734]: osdmap e159: 8 total, 8 up, 8 in 2024-03-24T12:13:35.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:13:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:13:36.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:35 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:36.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:35 smithi090 ceph-mon[33646]: osdmap e160: 8 total, 8 up, 8 in 2024-03-24T12:13:36.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:35 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/337824239' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi006-81775-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:36.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:35 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2470384770' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:36.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:35 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1568076090' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi006-81794-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:36.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:35 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1794130133' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi006-82056-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:36.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:36.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi006-81775-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:36.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi006-81794-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:36.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi006-82056-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:36.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:35 smithi090 ceph-mon[33646]: pgmap v198: 420 pgs: 96 unknown, 324 active+clean; 592 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:13:36.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:35 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:36.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]': finished 2024-03-24T12:13:36.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi006-81775-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:36.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi006-81794-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:36.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi006-82056-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:36.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:35 smithi090 ceph-mon[33646]: osdmap e161: 8 total, 8 up, 8 in 2024-03-24T12:13:36.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:35 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:36.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:35 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2470384770' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:36.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:36.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:36.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[30108]: osdmap e160: 8 total, 8 up, 8 in 2024-03-24T12:13:36.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/337824239' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi006-81775-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:36.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2470384770' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:36.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1568076090' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi006-81794-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:36.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1794130133' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi006-82056-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:36.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:36.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi006-81775-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:36.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi006-81794-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:36.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi006-82056-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:36.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[30108]: pgmap v198: 420 pgs: 96 unknown, 324 active+clean; 592 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:13:36.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:36.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]': finished 2024-03-24T12:13:36.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi006-81775-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:36.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi006-81794-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:36.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi006-82056-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:36.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[30108]: osdmap e161: 8 total, 8 up, 8 in 2024-03-24T12:13:36.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:36.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2470384770' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:36.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:36.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:36.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[41734]: osdmap e160: 8 total, 8 up, 8 in 2024-03-24T12:13:36.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/337824239' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi006-81775-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:36.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2470384770' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:36.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1568076090' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi006-81794-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:36.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1794130133' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi006-82056-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:36.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:36.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi006-81775-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:36.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi006-81794-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:36.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi006-82056-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:36.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[41734]: pgmap v198: 420 pgs: 96 unknown, 324 active+clean; 592 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:13:36.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:36.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi006-82137-16"}]': finished 2024-03-24T12:13:36.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi006-81775-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:36.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi006-81794-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:36.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi006-82056-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:36.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[41734]: osdmap e161: 8 total, 8 up, 8 in 2024-03-24T12:13:36.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:36.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2470384770' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:36.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi006-82137-16"}]: dispatch 2024-03-24T12:13:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:37 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi006-82137-16"}]': finished 2024-03-24T12:13:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:37 smithi090 ceph-mon[33646]: osdmap e162: 8 total, 8 up, 8 in 2024-03-24T12:13:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1072945251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/855880461' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:37 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1072945251' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:37 smithi090 ceph-mon[33646]: pgmap v201: 324 pgs: 324 active+clean; 592 KiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:13:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:37 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-24T12:13:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:37 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:37 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/855880461' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1072945251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:37 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-24T12:13:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:37 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:38.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:37 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:38.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/855880461' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi006-81775-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:38.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:37 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi006-81775-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:38.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi006-82137-16"}]': finished 2024-03-24T12:13:38.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[30108]: osdmap e162: 8 total, 8 up, 8 in 2024-03-24T12:13:38.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:38.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1072945251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:38.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/855880461' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:38.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:38.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1072945251' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:38.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[30108]: pgmap v201: 324 pgs: 324 active+clean; 592 KiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:13:38.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-24T12:13:38.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:38.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:38.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/855880461' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:38.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1072945251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:38.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-24T12:13:38.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:38.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:38.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/855880461' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi006-81775-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:38.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi006-81775-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:38.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi006-82137-16"}]': finished 2024-03-24T12:13:38.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[41734]: osdmap e162: 8 total, 8 up, 8 in 2024-03-24T12:13:38.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:38.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1072945251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:38.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/855880461' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:38.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:38.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1072945251' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:38.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[41734]: pgmap v201: 324 pgs: 324 active+clean; 592 KiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:13:38.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-24T12:13:38.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:38.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:38.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/855880461' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:38.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1072945251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:38.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-24T12:13:38.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:38.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:38.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/855880461' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi006-81775-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:38.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:37 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi006-81775-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:39.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:39 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-03-24T12:13:39.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:39.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi006-81775-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:39.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:39 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:39.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:39 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/855880461' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi006-81775-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:39.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:39 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1072945251' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:39.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:39 smithi090 ceph-mon[33646]: osdmap e163: 8 total, 8 up, 8 in 2024-03-24T12:13:39.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:39 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1193301302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi006-82056-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:39.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:39 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3590887818' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi006-81794-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:39.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi006-81775-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:39.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:39.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi006-82056-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:39.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi006-81794-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-03-24T12:13:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi006-81775-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/855880461' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi006-81775-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1072945251' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:39.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[30108]: osdmap e163: 8 total, 8 up, 8 in 2024-03-24T12:13:39.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1193301302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi006-82056-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:39.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3590887818' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi006-81794-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:39.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi006-81775-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:39.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:39.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi006-82056-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:39.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi006-81794-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:39.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-03-24T12:13:39.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:39.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi006-81775-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:39.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:39.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/855880461' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi006-81775-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:39.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1072945251' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:39.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[41734]: osdmap e163: 8 total, 8 up, 8 in 2024-03-24T12:13:39.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1193301302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi006-82056-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:39.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3590887818' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi006-81794-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:39.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi006-81775-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:39.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:39.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi006-82056-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:39.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi006-81794-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:40.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:40 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi006-82056-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:40.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:40 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi006-81794-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:40.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:40 smithi090 ceph-mon[33646]: pgmap v204: 388 pgs: 64 unknown, 324 active+clean; 592 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:13:40.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:40.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pg_num","val":"11"}]: dispatch 2024-03-24T12:13:40.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:40 smithi090 ceph-mon[33646]: osdmap e164: 8 total, 8 up, 8 in 2024-03-24T12:13:40.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:40 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pg_num","val":"11"}]: dispatch 2024-03-24T12:13:40.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:40 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi006-81775-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi006-81775-27"}]': finished 2024-03-24T12:13:40.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:40 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]': finished 2024-03-24T12:13:40.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:40 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pg_num","val":"11"}]': finished 2024-03-24T12:13:40.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:40 smithi090 ceph-mon[33646]: osdmap e165: 8 total, 8 up, 8 in 2024-03-24T12:13:40.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:40.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/35906651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:40.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:40 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:40.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/35906651' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:40.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:40 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:40.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/35906651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:40.681 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:40 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi006-82056-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi006-81794-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[30108]: pgmap v204: 388 pgs: 64 unknown, 324 active+clean; 592 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:13:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:40.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pg_num","val":"11"}]: dispatch 2024-03-24T12:13:40.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[30108]: osdmap e164: 8 total, 8 up, 8 in 2024-03-24T12:13:40.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pg_num","val":"11"}]: dispatch 2024-03-24T12:13:40.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi006-81775-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi006-81775-27"}]': finished 2024-03-24T12:13:40.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]': finished 2024-03-24T12:13:40.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pg_num","val":"11"}]': finished 2024-03-24T12:13:40.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[30108]: osdmap e165: 8 total, 8 up, 8 in 2024-03-24T12:13:40.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:40.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/35906651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:40.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:40.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/35906651' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:40.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:40.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/35906651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:40.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:40.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi006-82056-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:40.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi006-81794-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:40.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[41734]: pgmap v204: 388 pgs: 64 unknown, 324 active+clean; 592 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:13:40.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:40.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pg_num","val":"11"}]: dispatch 2024-03-24T12:13:40.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[41734]: osdmap e164: 8 total, 8 up, 8 in 2024-03-24T12:13:40.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pg_num","val":"11"}]: dispatch 2024-03-24T12:13:40.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi006-81775-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi006-81775-27"}]': finished 2024-03-24T12:13:40.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]': finished 2024-03-24T12:13:40.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pg_num","val":"11"}]': finished 2024-03-24T12:13:40.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[41734]: osdmap e165: 8 total, 8 up, 8 in 2024-03-24T12:13:40.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:40.715 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/35906651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:40.715 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:40.715 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/35906651' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:40.715 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:40.715 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/35906651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:40.715 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:40 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:41.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:41 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:41.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:41 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:41.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:41 smithi090 ceph-mon[33646]: osdmap e166: 8 total, 8 up, 8 in 2024-03-24T12:13:41.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:41.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/423530057' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi006-81794-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:41.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/35906651' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi006-82056-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:41.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:41 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi006-81794-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:41.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:41 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi006-82056-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:41.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:41 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:41.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:41 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:41.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:41 smithi006 ceph-mon[30108]: osdmap e166: 8 total, 8 up, 8 in 2024-03-24T12:13:41.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:41.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/423530057' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi006-81794-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:41.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/35906651' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi006-82056-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:41.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:41 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi006-81794-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:41.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:41 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi006-82056-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:41.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:41 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:41.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:41 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:41.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:41 smithi006 ceph-mon[41734]: osdmap e166: 8 total, 8 up, 8 in 2024-03-24T12:13:41.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:41.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/423530057' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi006-81794-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:41.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/35906651' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi006-82056-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:41.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:41 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi006-81794-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:41.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:41 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi006-82056-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:42.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:42 smithi090 ceph-mon[33646]: pgmap v207: 372 pgs: 48 unknown, 324 active+clean; 592 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:13:42.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:42 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi006-81794-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:42.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:42 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:42.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:42 smithi090 ceph-mon[33646]: osdmap e167: 8 total, 8 up, 8 in 2024-03-24T12:13:42.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:42 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/855880461' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:42.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:42 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:42.679 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:13:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:13:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:42 smithi006 ceph-mon[30108]: pgmap v207: 372 pgs: 48 unknown, 324 active+clean; 592 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:13:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:42 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi006-81794-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:42 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:42 smithi006 ceph-mon[30108]: osdmap e167: 8 total, 8 up, 8 in 2024-03-24T12:13:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:42 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/855880461' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:42 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:42.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:42 smithi006 ceph-mon[41734]: pgmap v207: 372 pgs: 48 unknown, 324 active+clean; 592 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:13:42.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:42 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi006-81794-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:42.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:42 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:42.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:42 smithi006 ceph-mon[41734]: osdmap e167: 8 total, 8 up, 8 in 2024-03-24T12:13:42.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:42 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/855880461' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:42.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:42 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:43.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:43 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:13:43.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:43 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi006-82056-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi006-82056-36"}]': finished 2024-03-24T12:13:43.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:43 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi006-81775-27"}]': finished 2024-03-24T12:13:43.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:43 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:43.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:43 smithi090 ceph-mon[33646]: osdmap e168: 8 total, 8 up, 8 in 2024-03-24T12:13:43.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:43 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/855880461' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:43.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:43 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:43.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:43 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-03-24T12:13:43.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:43 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-03-24T12:13:43.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:43 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:13:43.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:43 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi006-82056-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi006-82056-36"}]': finished 2024-03-24T12:13:43.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:43 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi006-81775-27"}]': finished 2024-03-24T12:13:43.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:43 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:43.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:43 smithi006 ceph-mon[30108]: osdmap e168: 8 total, 8 up, 8 in 2024-03-24T12:13:43.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:43 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/855880461' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:43.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:43 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:43.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:43 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-03-24T12:13:43.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:43 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-03-24T12:13:43.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:43 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:13:43.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:43 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi006-82056-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi006-82056-36"}]': finished 2024-03-24T12:13:43.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:43 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi006-81775-27"}]': finished 2024-03-24T12:13:43.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:43 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:43.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:43 smithi006 ceph-mon[41734]: osdmap e168: 8 total, 8 up, 8 in 2024-03-24T12:13:43.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:43 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/855880461' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:43.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:43 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi006-81775-27"}]: dispatch 2024-03-24T12:13:43.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:43 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-03-24T12:13:43.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:43 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-03-24T12:13:43.935 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: Running main() from gmock_main.cc 2024-03-24T12:13:43.935 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2024-03-24T12:13:43.935 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [----------] Global test environment set-up. 2024-03-24T12:13:43.935 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [----------] 26 tests from LibRadosAio 2024-03-24T12:13:43.935 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-03-24T12:13:43.935 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.TooBig (11096 ms) 2024-03-24T12:13:43.935 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-03-24T12:13:43.935 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (9185 ms) 2024-03-24T12:13:43.935 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-03-24T12:13:43.935 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (3579 ms) 2024-03-24T12:13:43.935 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-03-24T12:13:43.935 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (4708 ms) 2024-03-24T12:13:43.935 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-03-24T12:13:43.936 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (3966 ms) 2024-03-24T12:13:43.936 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-03-24T12:13:43.936 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (2988 ms) 2024-03-24T12:13:43.936 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-03-24T12:13:43.936 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (3083 ms) 2024-03-24T12:13:43.936 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-03-24T12:13:43.936 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (2842 ms) 2024-03-24T12:13:43.936 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-03-24T12:13:43.936 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (3002 ms) 2024-03-24T12:13:43.936 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-03-24T12:13:43.936 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (3005 ms) 2024-03-24T12:13:43.936 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-03-24T12:13:43.936 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (3016 ms) 2024-03-24T12:13:43.936 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-03-24T12:13:43.936 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (3005 ms) 2024-03-24T12:13:43.937 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-03-24T12:13:43.937 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (2988 ms) 2024-03-24T12:13:43.937 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-03-24T12:13:43.937 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (2996 ms) 2024-03-24T12:13:43.937 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-03-24T12:13:43.937 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.Flush (3011 ms) 2024-03-24T12:13:43.937 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-03-24T12:13:43.937 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (2986 ms) 2024-03-24T12:13:43.937 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-03-24T12:13:43.937 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (3014 ms) 2024-03-24T12:13:43.937 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-03-24T12:13:43.937 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (2988 ms) 2024-03-24T12:13:43.937 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-03-24T12:13:43.937 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3016 ms) 2024-03-24T12:13:43.937 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-03-24T12:13:43.937 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3016 ms) 2024-03-24T12:13:43.938 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2024-03-24T12:13:43.938 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (2972 ms) 2024-03-24T12:13:43.938 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-03-24T12:13:43.938 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3011 ms) 2024-03-24T12:13:43.938 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-03-24T12:13:43.938 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (3017 ms) 2024-03-24T12:13:43.938 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-03-24T12:13:43.938 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3021 ms) 2024-03-24T12:13:43.938 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-03-24T12:13:43.938 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (3011 ms) 2024-03-24T12:13:43.938 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-03-24T12:13:43.938 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (3009 ms) 2024-03-24T12:13:43.938 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [----------] 26 tests from LibRadosAio (95531 ms total) 2024-03-24T12:13:43.938 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: 2024-03-24T12:13:43.938 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-03-24T12:13:43.939 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-03-24T12:13:44.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:44 smithi090 ceph-mon[33646]: pgmap v210: 340 pgs: 8 unknown, 332 active+clean; 597 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-03-24T12:13:44.682 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:44 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:44.682 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:44 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi006-81775-27"}]': finished 2024-03-24T12:13:44.682 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:44 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-03-24T12:13:44.682 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:44 smithi090 ceph-mon[33646]: osdmap e169: 8 total, 8 up, 8 in 2024-03-24T12:13:44.682 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:44 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3641224000' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi006-81794-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:44.682 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:44 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi006-81794-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:44.682 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:44 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/44916234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:44.682 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:44 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:44.682 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:44 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/44916234' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:44.682 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:44 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:44.682 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:44 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/44916234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi006-81775-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:44.683 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:44 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi006-81775-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:44.683 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:44 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:13:44.683 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:44 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:13:44.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[30108]: pgmap v210: 340 pgs: 8 unknown, 332 active+clean; 597 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-03-24T12:13:44.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:44.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi006-81775-27"}]': finished 2024-03-24T12:13:44.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-03-24T12:13:44.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[30108]: osdmap e169: 8 total, 8 up, 8 in 2024-03-24T12:13:44.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3641224000' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi006-81794-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:44.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi006-81794-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:44.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/44916234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:44.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:44.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/44916234' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:44.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:44.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/44916234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi006-81775-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:44.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi006-81775-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:44.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:13:44.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:13:44.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[41734]: pgmap v210: 340 pgs: 8 unknown, 332 active+clean; 597 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-03-24T12:13:44.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:44.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi006-81775-27"}]': finished 2024-03-24T12:13:44.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-03-24T12:13:44.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[41734]: osdmap e169: 8 total, 8 up, 8 in 2024-03-24T12:13:44.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3641224000' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi006-81794-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:44.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi006-81794-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:44.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/44916234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:44.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:44.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/44916234' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:44.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:44.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/44916234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi006-81775-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:44.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi006-81775-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:44.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:13:44.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:44 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:13:45.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:45.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi006-81794-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:45.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi006-81775-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:45.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:13:45.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:45 smithi090 ceph-mon[33646]: osdmap e170: 8 total, 8 up, 8 in 2024-03-24T12:13:45.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-13"}]: dispatch 2024-03-24T12:13:45.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/44916234' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi006-81775-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:45.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/35906651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:45.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-13"}]: dispatch 2024-03-24T12:13:45.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi006-81775-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:45.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:45.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:13:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:13:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:13:45.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:45.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi006-81794-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:45.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi006-81775-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:45.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:13:45.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:45 smithi006 ceph-mon[30108]: osdmap e170: 8 total, 8 up, 8 in 2024-03-24T12:13:45.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-13"}]: dispatch 2024-03-24T12:13:45.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/44916234' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi006-81775-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:45.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/35906651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:45.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-13"}]: dispatch 2024-03-24T12:13:45.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi006-81775-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:45.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:45.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:45.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi006-81794-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:45.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi006-81775-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:45.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:13:45.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:45 smithi006 ceph-mon[41734]: osdmap e170: 8 total, 8 up, 8 in 2024-03-24T12:13:45.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-13"}]: dispatch 2024-03-24T12:13:45.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/44916234' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi006-81775-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:45.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/35906651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:45.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-13"}]: dispatch 2024-03-24T12:13:45.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi006-81775-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:45.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:46.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:46 smithi090 ceph-mon[33646]: pgmap v213: 364 pgs: 32 unknown, 332 active+clean; 597 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-03-24T12:13:46.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:46 smithi090 ceph-mon[33646]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:46.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:46 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:13:46.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:46 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-13"}]': finished 2024-03-24T12:13:46.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:46 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36"}]': finished 2024-03-24T12:13:46.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:46 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/35906651' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:46.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:46 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:46.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:46 smithi090 ceph-mon[33646]: osdmap e171: 8 total, 8 up, 8 in 2024-03-24T12:13:46.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:46 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:46.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:46 smithi006 ceph-mon[30108]: pgmap v213: 364 pgs: 32 unknown, 332 active+clean; 597 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-03-24T12:13:46.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:46 smithi006 ceph-mon[30108]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:46.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:46 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:13:46.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:46 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-13"}]': finished 2024-03-24T12:13:46.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:46 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36"}]': finished 2024-03-24T12:13:46.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:46 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/35906651' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:46.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:46 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:46.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:46 smithi006 ceph-mon[30108]: osdmap e171: 8 total, 8 up, 8 in 2024-03-24T12:13:46.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:46 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:46.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:46 smithi006 ceph-mon[41734]: pgmap v213: 364 pgs: 32 unknown, 332 active+clean; 597 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-03-24T12:13:46.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:46 smithi006 ceph-mon[41734]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:46.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:46 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:13:46.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:46 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-13"}]': finished 2024-03-24T12:13:46.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:46 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi006-82056-36"}]': finished 2024-03-24T12:13:46.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:46 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/35906651' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:46.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:46 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:46.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:46 smithi006 ceph-mon[41734]: osdmap e171: 8 total, 8 up, 8 in 2024-03-24T12:13:46.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:46 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi006-82056-36"}]: dispatch 2024-03-24T12:13:46.829 INFO:tasks.workunit.client.0.smithi006.stdout:68032 ms total) 2024-03-24T12:13:46.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: 2024-03-24T12:13:46.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-03-24T12:13:46.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-03-24T12:13:46.829 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (137 ms) 2024-03-24T12:13:46.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (138 ms total) 2024-03-24T12:13:46.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: 2024-03-24T12:13:46.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-03-24T12:13:46.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-03-24T12:13:46.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (34 ms) 2024-03-24T12:13:46.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-03-24T12:13:46.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (10 ms) 2024-03-24T12:13:46.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (44 ms total) 2024-03-24T12:13:46.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: 2024-03-24T12:13:46.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-03-24T12:13:46.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-03-24T12:13:46.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (53 ms) 2024-03-24T12:13:46.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-03-24T12:13:46.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (5 ms) 2024-03-24T12:13:46.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (59 ms total) 2024-03-24T12:13:46.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: 2024-03-24T12:13:46.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-03-24T12:13:46.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-03-24T12:13:46.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (291 ms) 2024-03-24T12:13:46.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-03-24T12:13:46.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (7 ms) 2024-03-24T12:13:46.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (298 ms total) 2024-03-24T12:13:46.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: 2024-03-24T12:13:46.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-03-24T12:13:46.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-03-24T12:13:46.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1027 ms) 2024-03-24T12:13:46.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1027 ms total) 2024-03-24T12:13:46.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: 2024-03-24T12:13:46.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-03-24T12:13:46.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (105323 ms total) 2024-03-24T12:13:46.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-03-24T12:13:48.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:47 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi006-81775-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi006-81775-28"}]': finished 2024-03-24T12:13:48.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:47 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi006-82056-36"}]': finished 2024-03-24T12:13:48.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:47 smithi090 ceph-mon[33646]: osdmap e172: 8 total, 8 up, 8 in 2024-03-24T12:13:48.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1355399458' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi006-81794-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:48.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:48.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:47 smithi090 ceph-mon[33646]: pgmap v216: 339 pgs: 10 creating+peering, 30 unknown, 299 active+clean; 595 KiB data, 597 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:13:48.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:47 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi006-81794-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:48.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:47 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:13:48.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:47 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:13:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:47 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi006-81775-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi006-81775-28"}]': finished 2024-03-24T12:13:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:47 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi006-82056-36"}]': finished 2024-03-24T12:13:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:47 smithi006 ceph-mon[30108]: osdmap e172: 8 total, 8 up, 8 in 2024-03-24T12:13:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1355399458' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi006-81794-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:47 smithi006 ceph-mon[30108]: pgmap v216: 339 pgs: 10 creating+peering, 30 unknown, 299 active+clean; 595 KiB data, 597 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:13:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:47 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi006-81794-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:48.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:47 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:13:48.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:47 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:13:48.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:47 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi006-81775-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi006-81775-28"}]': finished 2024-03-24T12:13:48.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:47 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi006-82056-36"}]': finished 2024-03-24T12:13:48.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:47 smithi006 ceph-mon[41734]: osdmap e172: 8 total, 8 up, 8 in 2024-03-24T12:13:48.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1355399458' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi006-81794-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:48.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:48.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:47 smithi006 ceph-mon[41734]: pgmap v216: 339 pgs: 10 creating+peering, 30 unknown, 299 active+clean; 595 KiB data, 597 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:13:48.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:47 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi006-81794-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:48.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:47 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:13:48.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:47 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:13:49.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:48 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi006-81794-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:49.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:48 smithi090 ceph-mon[33646]: osdmap e173: 8 total, 8 up, 8 in 2024-03-24T12:13:49.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:49.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:48 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:49.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:48 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi006-81794-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:48 smithi006 ceph-mon[30108]: osdmap e173: 8 total, 8 up, 8 in 2024-03-24T12:13:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:48 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:49.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:48 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi006-81794-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:49.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:48 smithi006 ceph-mon[41734]: osdmap e173: 8 total, 8 up, 8 in 2024-03-24T12:13:49.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:49.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:48 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:49.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:50.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:49 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:50.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:49 smithi090 ceph-mon[33646]: osdmap e174: 8 total, 8 up, 8 in 2024-03-24T12:13:50.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:49 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/44916234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:50.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:49 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:50.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:49 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:50.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:49 smithi090 ceph-mon[33646]: pgmap v219: 331 pgs: 32 unknown, 299 active+clean; 609 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-03-24T12:13:50.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:49 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:50.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:49 smithi006 ceph-mon[30108]: osdmap e174: 8 total, 8 up, 8 in 2024-03-24T12:13:50.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:49 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/44916234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:50.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:49 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:50.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:49 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:50.215 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:49 smithi006 ceph-mon[30108]: pgmap v219: 331 pgs: 32 unknown, 299 active+clean; 609 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-03-24T12:13:50.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:49 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:50.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:49 smithi006 ceph-mon[41734]: osdmap e174: 8 total, 8 up, 8 in 2024-03-24T12:13:50.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:49 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/44916234' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:50.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:49 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:50.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:49 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:50.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:49 smithi006 ceph-mon[41734]: pgmap v219: 331 pgs: 32 unknown, 299 active+clean; 609 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-03-24T12:13:51.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:50 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi006-81775-28"}]': finished 2024-03-24T12:13:51.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:50 smithi090 ceph-mon[33646]: osdmap e175: 8 total, 8 up, 8 in 2024-03-24T12:13:51.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:50 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/44916234' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:51.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:50 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:51.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:50 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3932324053' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi006-81794-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:51.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:50 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi006-81794-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:51.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:50 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:13:51.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:50 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:13:51.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:50 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:51.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:50 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:51.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:50 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi006-81775-28"}]': finished 2024-03-24T12:13:51.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:50 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi006-81794-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:51.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:50 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-15", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:13:51.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:50 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-15"}]: dispatch 2024-03-24T12:13:51.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:50 smithi090 ceph-mon[33646]: osdmap e176: 8 total, 8 up, 8 in 2024-03-24T12:13:51.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:50 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-15"}]: dispatch 2024-03-24T12:13:51.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi006-81775-28"}]': finished 2024-03-24T12:13:51.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[30108]: osdmap e175: 8 total, 8 up, 8 in 2024-03-24T12:13:51.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/44916234' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:51.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:51.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3932324053' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi006-81794-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:51.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi006-81794-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:51.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:13:51.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:13:51.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:51.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:51.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi006-81775-28"}]': finished 2024-03-24T12:13:51.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi006-81794-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:51.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-15", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:13:51.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-15"}]: dispatch 2024-03-24T12:13:51.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[30108]: osdmap e176: 8 total, 8 up, 8 in 2024-03-24T12:13:51.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-15"}]: dispatch 2024-03-24T12:13:51.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi006-81775-28"}]': finished 2024-03-24T12:13:51.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[41734]: osdmap e175: 8 total, 8 up, 8 in 2024-03-24T12:13:51.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/44916234' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:51.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi006-81775-28"}]: dispatch 2024-03-24T12:13:51.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3932324053' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi006-81794-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:51.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi006-81794-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:51.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:13:51.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:13:51.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:51.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:51.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi006-81775-28"}]': finished 2024-03-24T12:13:51.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi006-81794-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:51.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-15", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:13:51.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-15"}]: dispatch 2024-03-24T12:13:51.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[41734]: osdmap e176: 8 total, 8 up, 8 in 2024-03-24T12:13:51.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:50 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-15"}]: dispatch 2024-03-24T12:13:51.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-03-24T12:13:51.830 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-03-24T12:13:51.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-03-24T12:13:51.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-03-24T12:13:51.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-03-24T12:13:51.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (11094 ms) 2024-03-24T12:13:51.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-03-24T12:13:51.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (21440 ms) 2024-03-24T12:13:51.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-03-24T12:13:51.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (6084 ms) 2024-03-24T12:13:51.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-03-24T12:13:51.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (2832 ms) 2024-03-24T12:13:51.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-03-24T12:13:51.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (3005 ms) 2024-03-24T12:13:51.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-03-24T12:13:51.831 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (2997 ms) 2024-03-24T12:13:51.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-03-24T12:13:51.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (3015 ms) 2024-03-24T12:13:51.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-03-24T12:13:51.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (3005 ms) 2024-03-24T12:13:51.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-03-24T12:13:51.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (2983 ms) 2024-03-24T12:13:51.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-03-24T12:13:51.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (2995 ms) 2024-03-24T12:13:51.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-03-24T12:13:51.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (3006 ms) 2024-03-24T12:13:51.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-03-24T12:13:51.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (2997 ms) 2024-03-24T12:13:51.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-03-24T12:13:51.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3014 ms) 2024-03-24T12:13:51.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-03-24T12:13:51.832 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (2995 ms) 2024-03-24T12:13:51.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-03-24T12:13:51.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3007 ms) 2024-03-24T12:13:51.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-03-24T12:13:51.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3029 ms) 2024-03-24T12:13:51.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-03-24T12:13:51.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (2959 ms) 2024-03-24T12:13:51.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-03-24T12:13:51.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (3017 ms) 2024-03-24T12:13:51.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-03-24T12:13:51.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3013 ms) 2024-03-24T12:13:51.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-03-24T12:13:51.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3015 ms) 2024-03-24T12:13:51.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-03-24T12:13:51.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3015 ms) 2024-03-24T12:13:51.833 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-03-24T12:13:51.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3011 ms) 2024-03-24T12:13:51.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-03-24T12:13:51.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3022 ms) 2024-03-24T12:13:51.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-03-24T12:13:51.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (3013 ms) 2024-03-24T12:13:51.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-03-24T12:13:51.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (3146 ms) 2024-03-24T12:13:51.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-03-24T12:13:51.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (2767 ms) 2024-03-24T12:13:51.834 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-03-24T12:13:52.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:51 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1846772046' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:52.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:51 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:52.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:51 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1846772046' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:52.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:51 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:52.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:51 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1846772046' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi006-81775-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:52.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:51 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi006-81775-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:52.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:51 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:52.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:51 smithi090 ceph-mon[33646]: pgmap v222: 363 pgs: 64 unknown, 299 active+clean; 609 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 6 op/s; 31 B/s, 0 objects/s recovering 2024-03-24T12:13:52.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:51 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-15"}]': finished 2024-03-24T12:13:52.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:51 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi006-81775-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:52.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:51 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-15", "mode": "writeback"}]: dispatch 2024-03-24T12:13:52.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:51 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1846772046' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi006-81775-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:52.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:51 smithi090 ceph-mon[33646]: osdmap e177: 8 total, 8 up, 8 in 2024-03-24T12:13:52.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:51 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-15", "mode": "writeback"}]: dispatch 2024-03-24T12:13:52.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:51 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi006-81775-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:52.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1846772046' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:52.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:52.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1846772046' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:52.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:52.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1846772046' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi006-81775-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:52.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi006-81775-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:52.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:52.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[30108]: pgmap v222: 363 pgs: 64 unknown, 299 active+clean; 609 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 6 op/s; 31 B/s, 0 objects/s recovering 2024-03-24T12:13:52.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-15"}]': finished 2024-03-24T12:13:52.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi006-81775-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:52.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-15", "mode": "writeback"}]: dispatch 2024-03-24T12:13:52.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1846772046' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi006-81775-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:52.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[30108]: osdmap e177: 8 total, 8 up, 8 in 2024-03-24T12:13:52.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-15", "mode": "writeback"}]: dispatch 2024-03-24T12:13:52.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi006-81775-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:52.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1846772046' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:52.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:52.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1846772046' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:52.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:52.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1846772046' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi006-81775-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:52.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi006-81775-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:52.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:52.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[41734]: pgmap v222: 363 pgs: 64 unknown, 299 active+clean; 609 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 6 op/s; 31 B/s, 0 objects/s recovering 2024-03-24T12:13:52.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-15"}]': finished 2024-03-24T12:13:52.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi006-81775-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:13:52.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-15", "mode": "writeback"}]: dispatch 2024-03-24T12:13:52.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1846772046' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi006-81775-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:52.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[41734]: osdmap e177: 8 total, 8 up, 8 in 2024-03-24T12:13:52.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-15", "mode": "writeback"}]: dispatch 2024-03-24T12:13:52.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:51 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi006-81775-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:52.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:13:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:13:53.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:53 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:53.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:53 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:13:53.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:53 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:13:53.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:53 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-15", "mode": "writeback"}]': finished 2024-03-24T12:13:53.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:53 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:53.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:53 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:13:53.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:53 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:13:53.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:53 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-15", "mode": "writeback"}]': finished 2024-03-24T12:13:53.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:53 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:53.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:53 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:13:53.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:53 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:13:53.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:53 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-15", "mode": "writeback"}]': finished 2024-03-24T12:13:54.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:54.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:54 smithi090 ceph-mon[33646]: pgmap v225: 355 pgs: 32 unknown, 323 active+clean; 585 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:13:54.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:54 smithi090 ceph-mon[33646]: osdmap e178: 8 total, 8 up, 8 in 2024-03-24T12:13:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1072945251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:54 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1053435496' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi006-81794-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:54 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:13:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:54 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi006-81775-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi006-81775-29"}]': finished 2024-03-24T12:13:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:54 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]': finished 2024-03-24T12:13:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1053435496' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi006-81794-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:54 smithi090 ceph-mon[33646]: osdmap e179: 8 total, 8 up, 8 in 2024-03-24T12:13:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1072945251' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:54 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:13:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:54 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:13:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:54 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:13:54.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:54 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:13:54.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:54 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:13:54.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:54.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[41734]: pgmap v225: 355 pgs: 32 unknown, 323 active+clean; 585 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:13:54.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[41734]: osdmap e178: 8 total, 8 up, 8 in 2024-03-24T12:13:54.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1072945251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:54.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:54.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1053435496' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi006-81794-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:54.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:13:54.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi006-81775-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi006-81775-29"}]': finished 2024-03-24T12:13:54.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]': finished 2024-03-24T12:13:54.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1053435496' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi006-81794-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:54.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[41734]: osdmap e179: 8 total, 8 up, 8 in 2024-03-24T12:13:54.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1072945251' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:54.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:54.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:13:54.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:13:54.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:54.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:13:54.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:13:54.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:13:54.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:54.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[30108]: pgmap v225: 355 pgs: 32 unknown, 323 active+clean; 585 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:13:54.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[30108]: osdmap e178: 8 total, 8 up, 8 in 2024-03-24T12:13:54.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1072945251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:54.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:54.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1053435496' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi006-81794-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:54.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:13:54.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi006-81775-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi006-81775-29"}]': finished 2024-03-24T12:13:54.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]': finished 2024-03-24T12:13:54.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1053435496' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi006-81794-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:54.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[30108]: osdmap e179: 8 total, 8 up, 8 in 2024-03-24T12:13:54.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1072945251' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:54.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]: dispatch 2024-03-24T12:13:54.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:13:54.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:13:54.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:54.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:13:54.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:13:54.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:54 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:13:54.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAinapshotsECPP.SnapRemovePP (2007 ms) 2024-03-24T12:13:54.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-03-24T12:13:54.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2016 ms) 2024-03-24T12:13:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-03-24T12:13:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (2008 ms) 2024-03-24T12:13:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (9053 ms total) 2024-03-24T12:13:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: 2024-03-24T12:13:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-03-24T12:13:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-03-24T12:13:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4054 ms) 2024-03-24T12:13:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-03-24T12:13:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3876 ms) 2024-03-24T12:13:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-03-24T12:13:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (3993 ms) 2024-03-24T12:13:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (11923 ms total) 2024-03-24T12:13:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: 2024-03-24T12:13:54.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-03-24T12:13:54.844 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [==========] 21 tests from 5 test suites ran. (113338 ms total) 2024-03-24T12:13:54.844 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-03-24T12:13:54.844 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2024-03-24T12:13:54.844 INFO:tasks.workunit.client.0.smithi006.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-03-24T12:13:55.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:13:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:13:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:13:56.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:55 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]': finished 2024-03-24T12:13:56.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:55 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:13:56.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:55 smithi090 ceph-mon[33646]: osdmap e180: 8 total, 8 up, 8 in 2024-03-24T12:13:56.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:55 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-15"}]: dispatch 2024-03-24T12:13:56.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:55 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-15"}]: dispatch 2024-03-24T12:13:56.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:55 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:56.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:55 smithi090 ceph-mon[33646]: pgmap v228: 331 pgs: 8 unknown, 323 active+clean; 585 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:13:56.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:55 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]': finished 2024-03-24T12:13:56.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:55 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:13:56.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:55 smithi006 ceph-mon[41734]: osdmap e180: 8 total, 8 up, 8 in 2024-03-24T12:13:56.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:55 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-15"}]: dispatch 2024-03-24T12:13:56.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:55 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-15"}]: dispatch 2024-03-24T12:13:56.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:55 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:56.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:55 smithi006 ceph-mon[41734]: pgmap v228: 331 pgs: 8 unknown, 323 active+clean; 585 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:13:56.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:55 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi006-82137-21"}]': finished 2024-03-24T12:13:56.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:55 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:13:56.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:55 smithi006 ceph-mon[30108]: osdmap e180: 8 total, 8 up, 8 in 2024-03-24T12:13:56.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:55 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-15"}]: dispatch 2024-03-24T12:13:56.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:55 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-15"}]: dispatch 2024-03-24T12:13:56.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:55 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:56.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:55 smithi006 ceph-mon[30108]: pgmap v228: 331 pgs: 8 unknown, 323 active+clean; 585 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:13:57.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:56 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:13:57.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:56 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-15"}]': finished 2024-03-24T12:13:57.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:56 smithi090 ceph-mon[33646]: osdmap e181: 8 total, 8 up, 8 in 2024-03-24T12:13:57.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:56 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1846772046' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:57.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:56 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:57.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:56 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1651201218' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi006-81794-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:57.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:56 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:57.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:56 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:13:57.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:56 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-15"}]': finished 2024-03-24T12:13:57.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:56 smithi006 ceph-mon[30108]: osdmap e181: 8 total, 8 up, 8 in 2024-03-24T12:13:57.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:56 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1846772046' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:57.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:56 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:57.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:56 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1651201218' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi006-81794-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:57.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:56 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:57.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:56 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:13:57.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:56 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-15"}]': finished 2024-03-24T12:13:57.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:56 smithi006 ceph-mon[41734]: osdmap e181: 8 total, 8 up, 8 in 2024-03-24T12:13:57.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:56 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1846772046' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:57.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:56 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:57.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:56 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1651201218' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi006-81794-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:57.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:56 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:58.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:57 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi006-81775-29"}]': finished 2024-03-24T12:13:58.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:57 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1651201218' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi006-81794-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:58.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:57 smithi090 ceph-mon[33646]: osdmap e182: 8 total, 8 up, 8 in 2024-03-24T12:13:58.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:57 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1846772046' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:58.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:57 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:58.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:57 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:58.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:57 smithi090 ceph-mon[33646]: pgmap v231: 323 pgs: 16 unknown, 307 active+clean; 585 KiB data, 587 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:13:58.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:57 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi006-81775-29"}]': finished 2024-03-24T12:13:58.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:57 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1651201218' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi006-81794-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:58.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:57 smithi006 ceph-mon[30108]: osdmap e182: 8 total, 8 up, 8 in 2024-03-24T12:13:58.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:57 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1846772046' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:58.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:57 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:58.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:57 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:58.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:57 smithi006 ceph-mon[30108]: pgmap v231: 323 pgs: 16 unknown, 307 active+clean; 585 KiB data, 587 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:13:58.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:57 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi006-81775-29"}]': finished 2024-03-24T12:13:58.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:57 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1651201218' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi006-81794-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:13:58.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:57 smithi006 ceph-mon[41734]: osdmap e182: 8 total, 8 up, 8 in 2024-03-24T12:13:58.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:57 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1846772046' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:58.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:57 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi006-81775-29"}]: dispatch 2024-03-24T12:13:58.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:57 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:58.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:57 smithi006 ceph-mon[41734]: pgmap v231: 323 pgs: 16 unknown, 307 active+clean; 585 KiB data, 587 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:13:59.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:58 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:59.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:58 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi006-81775-29"}]': finished 2024-03-24T12:13:59.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:58 smithi090 ceph-mon[33646]: osdmap e183: 8 total, 8 up, 8 in 2024-03-24T12:13:59.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:58 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:59.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:58 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:59.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:58 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi006-81775-30"}]: dispatch 2024-03-24T12:13:59.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:58 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:59.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:58 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi006-81775-30"}]: dispatch 2024-03-24T12:13:59.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:58 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi006-81775-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:59.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:58 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:59.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:58 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi006-81775-29"}]': finished 2024-03-24T12:13:59.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:58 smithi006 ceph-mon[30108]: osdmap e183: 8 total, 8 up, 8 in 2024-03-24T12:13:59.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:58 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:59.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:58 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:59.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:58 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi006-81775-30"}]: dispatch 2024-03-24T12:13:59.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:58 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:59.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:58 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi006-81775-30"}]: dispatch 2024-03-24T12:13:59.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:58 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi006-81775-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:13:59.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:58 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:13:59.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:58 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi006-81775-29"}]': finished 2024-03-24T12:13:59.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:58 smithi006 ceph-mon[41734]: osdmap e183: 8 total, 8 up, 8 in 2024-03-24T12:13:59.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:58 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:59.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:58 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:13:59.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:58 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi006-81775-30"}]: dispatch 2024-03-24T12:13:59.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:58 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:13:59.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:58 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi006-81775-30"}]: dispatch 2024-03-24T12:13:59.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:58 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi006-81775-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:00.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:59 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:00.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:59 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi006-81775-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:00.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:59 smithi090 ceph-mon[33646]: osdmap e184: 8 total, 8 up, 8 in 2024-03-24T12:14:00.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:59 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3755263183' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi006-81794-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:00.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:59 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi006-81775-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi006-81775-30"}]: dispatch 2024-03-24T12:14:00.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:59 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi006-81794-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:00.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:59 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:00.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:59 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:00.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:59 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:00.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:13:59 smithi090 ceph-mon[33646]: pgmap v234: 355 pgs: 64 unknown, 291 active+clean; 585 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:14:00.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:59 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:00.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:59 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi006-81775-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:00.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:59 smithi006 ceph-mon[41734]: osdmap e184: 8 total, 8 up, 8 in 2024-03-24T12:14:00.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:59 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3755263183' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi006-81794-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:00.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:59 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi006-81775-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi006-81775-30"}]: dispatch 2024-03-24T12:14:00.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:59 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi006-81794-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:00.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:59 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:00.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:59 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:00.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:59 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:00.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:13:59 smithi006 ceph-mon[41734]: pgmap v234: 355 pgs: 64 unknown, 291 active+clean; 585 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:14:00.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:59 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:00.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:59 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi006-81775-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:00.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:59 smithi006 ceph-mon[30108]: osdmap e184: 8 total, 8 up, 8 in 2024-03-24T12:14:00.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:59 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3755263183' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi006-81794-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:00.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:59 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi006-81775-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi006-81775-30"}]: dispatch 2024-03-24T12:14:00.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:59 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi006-81794-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:00.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:59 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:00.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:59 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:00.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:59 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:00.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:13:59 smithi006 ceph-mon[30108]: pgmap v234: 355 pgs: 64 unknown, 291 active+clean; 585 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:14:01.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi006-81794-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:01.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-17", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:14:01.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-17"}]: dispatch 2024-03-24T12:14:01.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:01.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:01 smithi090 ceph-mon[33646]: osdmap e185: 8 total, 8 up, 8 in 2024-03-24T12:14:01.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-17"}]: dispatch 2024-03-24T12:14:01.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi006-81775-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi006-81775-30"}]': finished 2024-03-24T12:14:01.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-17"}]': finished 2024-03-24T12:14:01.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-17", "mode": "writeback"}]: dispatch 2024-03-24T12:14:01.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:01 smithi090 ceph-mon[33646]: osdmap e186: 8 total, 8 up, 8 in 2024-03-24T12:14:01.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-17", "mode": "writeback"}]: dispatch 2024-03-24T12:14:01.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:01.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi006-81794-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:01.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-17", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:14:01.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-17"}]: dispatch 2024-03-24T12:14:01.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:01.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[30108]: osdmap e185: 8 total, 8 up, 8 in 2024-03-24T12:14:01.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-17"}]: dispatch 2024-03-24T12:14:01.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi006-81775-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi006-81775-30"}]': finished 2024-03-24T12:14:01.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-17"}]': finished 2024-03-24T12:14:01.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-17", "mode": "writeback"}]: dispatch 2024-03-24T12:14:01.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[30108]: osdmap e186: 8 total, 8 up, 8 in 2024-03-24T12:14:01.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-17", "mode": "writeback"}]: dispatch 2024-03-24T12:14:01.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:01.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi006-81794-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:01.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-17", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:14:01.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-17"}]: dispatch 2024-03-24T12:14:01.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:01.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[41734]: osdmap e185: 8 total, 8 up, 8 in 2024-03-24T12:14:01.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-17"}]: dispatch 2024-03-24T12:14:01.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi006-81775-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi006-81775-30"}]': finished 2024-03-24T12:14:01.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-17"}]': finished 2024-03-24T12:14:01.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-17", "mode": "writeback"}]: dispatch 2024-03-24T12:14:01.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[41734]: osdmap e186: 8 total, 8 up, 8 in 2024-03-24T12:14:01.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-17", "mode": "writeback"}]: dispatch 2024-03-24T12:14:01.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:02.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:14:02 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:14:02.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:02 smithi090 ceph-mon[33646]: pgmap v237: 331 pgs: 40 unknown, 291 active+clean; 585 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:14:02.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:02 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:14:02.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:02 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-17", "mode": "writeback"}]': finished 2024-03-24T12:14:02.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:02 smithi090 ceph-mon[33646]: osdmap e187: 8 total, 8 up, 8 in 2024-03-24T12:14:02.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:02 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1061216982' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi006-81794-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:02.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:02 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:02.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:02 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:02.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:02 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:02.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:02 smithi006 ceph-mon[30108]: pgmap v237: 331 pgs: 40 unknown, 291 active+clean; 585 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:14:02.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:02 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:14:02.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:02 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-17", "mode": "writeback"}]': finished 2024-03-24T12:14:02.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:02 smithi006 ceph-mon[30108]: osdmap e187: 8 total, 8 up, 8 in 2024-03-24T12:14:02.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:02 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1061216982' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi006-81794-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:02.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:02 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:02.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:02 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:02.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:02 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:02.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:02 smithi006 ceph-mon[41734]: pgmap v237: 331 pgs: 40 unknown, 291 active+clean; 585 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:14:02.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:02 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:14:02.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:02 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-17", "mode": "writeback"}]': finished 2024-03-24T12:14:02.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:02 smithi006 ceph-mon[41734]: osdmap e187: 8 total, 8 up, 8 in 2024-03-24T12:14:02.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:02 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1061216982' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi006-81794-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:02.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:02 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:02.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:02 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:02.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:02 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:03.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:03 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:14:03.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:03 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:14:03.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:03 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:14:03.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1061216982' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi006-81794-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:03.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:03 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:14:03.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-17"}]: dispatch 2024-03-24T12:14:03.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:03 smithi090 ceph-mon[33646]: osdmap e188: 8 total, 8 up, 8 in 2024-03-24T12:14:03.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi006-81775-30"}]: dispatch 2024-03-24T12:14:03.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:03 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-17"}]: dispatch 2024-03-24T12:14:03.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:03.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:03 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:14:03.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:03 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:14:03.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:03 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:14:03.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1061216982' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi006-81794-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:03.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:03 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:14:03.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-17"}]: dispatch 2024-03-24T12:14:03.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:03 smithi006 ceph-mon[30108]: osdmap e188: 8 total, 8 up, 8 in 2024-03-24T12:14:03.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi006-81775-30"}]: dispatch 2024-03-24T12:14:03.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:03 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-17"}]: dispatch 2024-03-24T12:14:03.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:03.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:03 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:14:03.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:03 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:14:03.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:03 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:14:03.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1061216982' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi006-81794-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:03.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:03 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:14:03.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-17"}]: dispatch 2024-03-24T12:14:03.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:03 smithi006 ceph-mon[41734]: osdmap e188: 8 total, 8 up, 8 in 2024-03-24T12:14:03.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi006-81775-30"}]: dispatch 2024-03-24T12:14:03.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:03 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-17"}]: dispatch 2024-03-24T12:14:03.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:04.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:04 smithi090 ceph-mon[33646]: pgmap v240: 355 pgs: 32 creating+peering, 323 active+clean; 585 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-24T12:14:04.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:04 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1061216982' entity='client.admin' cmd=[{ 2024-03-24T12:14:04.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:04 smithi090 ceph-mon[33646]: "prefix": "osd pool set", 2024-03-24T12:14:04.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:04 smithi090 ceph-mon[33646]: "pool": "PoolEIOFlag_smithi006-81794-33", 2024-03-24T12:14:04.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:04 smithi090 ceph-mon[33646]: "var": "eio", 2024-03-24T12:14:04.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:04 smithi090 ceph-mon[33646]: "val": "true" 2024-03-24T12:14:04.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:04 smithi090 ceph-mon[33646]: }]: dispatch 2024-03-24T12:14:04.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:04 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:04.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:04 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:14:04.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:04 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi006-81775-30"}]': finished 2024-03-24T12:14:04.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:04 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-17"}]': finished 2024-03-24T12:14:04.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:04 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1061216982' entity='client.admin' cmd='[{ 2024-03-24T12:14:04.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:04 smithi090 ceph-mon[33646]: "prefix": "osd pool set", 2024-03-24T12:14:04.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:04 smithi090 ceph-mon[33646]: "pool": "PoolEIOFlag_smithi006-81794-33", 2024-03-24T12:14:04.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:04 smithi090 ceph-mon[33646]: "var": "eio", 2024-03-24T12:14:04.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:04 smithi090 ceph-mon[33646]: "val": "true" 2024-03-24T12:14:04.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:04 smithi090 ceph-mon[33646]: }]': finished 2024-03-24T12:14:04.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:04 smithi090 ceph-mon[33646]: osdmap e189: 8 total, 8 up, 8 in 2024-03-24T12:14:04.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:04 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi006-81775-30"}]: dispatch 2024-03-24T12:14:04.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:04 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:04.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[30108]: pgmap v240: 355 pgs: 32 creating+peering, 323 active+clean; 585 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-24T12:14:04.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1061216982' entity='client.admin' cmd=[{ 2024-03-24T12:14:04.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[30108]: "prefix": "osd pool set", 2024-03-24T12:14:04.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[30108]: "pool": "PoolEIOFlag_smithi006-81794-33", 2024-03-24T12:14:04.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[30108]: "var": "eio", 2024-03-24T12:14:04.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[30108]: "val": "true" 2024-03-24T12:14:04.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[30108]: }]: dispatch 2024-03-24T12:14:04.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:04.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:14:04.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi006-81775-30"}]': finished 2024-03-24T12:14:04.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-17"}]': finished 2024-03-24T12:14:04.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1061216982' entity='client.admin' cmd='[{ 2024-03-24T12:14:04.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[30108]: "prefix": "osd pool set", 2024-03-24T12:14:04.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[30108]: "pool": "PoolEIOFlag_smithi006-81794-33", 2024-03-24T12:14:04.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[30108]: "var": "eio", 2024-03-24T12:14:04.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[30108]: "val": "true" 2024-03-24T12:14:04.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[30108]: }]': finished 2024-03-24T12:14:04.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[30108]: osdmap e189: 8 total, 8 up, 8 in 2024-03-24T12:14:04.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi006-81775-30"}]: dispatch 2024-03-24T12:14:04.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:04.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[41734]: pgmap v240: 355 pgs: 32 creating+peering, 323 active+clean; 585 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-24T12:14:04.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1061216982' entity='client.admin' cmd=[{ 2024-03-24T12:14:04.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[41734]: "prefix": "osd pool set", 2024-03-24T12:14:04.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[41734]: "pool": "PoolEIOFlag_smithi006-81794-33", 2024-03-24T12:14:04.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[41734]: "var": "eio", 2024-03-24T12:14:04.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[41734]: "val": "true" 2024-03-24T12:14:04.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[41734]: }]: dispatch 2024-03-24T12:14:04.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:04.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:14:04.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi006-81775-30"}]': finished 2024-03-24T12:14:04.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-17"}]': finished 2024-03-24T12:14:04.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1061216982' entity='client.admin' cmd='[{ 2024-03-24T12:14:04.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[41734]: "prefix": "osd pool set", 2024-03-24T12:14:04.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[41734]: "pool": "PoolEIOFlag_smithi006-81794-33", 2024-03-24T12:14:04.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[41734]: "var": "eio", 2024-03-24T12:14:04.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[41734]: "val": "true" 2024-03-24T12:14:04.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[41734]: }]': finished 2024-03-24T12:14:04.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[41734]: osdmap e189: 8 total, 8 up, 8 in 2024-03-24T12:14:04.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi006-81775-30"}]: dispatch 2024-03-24T12:14:04.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:04 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:05.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:14:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:14:06.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:05 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi006-81775-30"}]': finished 2024-03-24T12:14:06.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:05 smithi090 ceph-mon[33646]: osdmap e190: 8 total, 8 up, 8 in 2024-03-24T12:14:06.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:05 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2179630808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:06.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:05 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:06.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:05 smithi090 ceph-mon[33646]: pgmap v243: 291 pgs: 291 active+clean; 585 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:14:06.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:05 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-03-24T12:14:06.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:05 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:06.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:05 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2179630808' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:06.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:05 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-03-24T12:14:06.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:05 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:06.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:05 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2179630808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi006-81775-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:06.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:05 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi006-81775-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:06.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi006-81775-30"}]': finished 2024-03-24T12:14:06.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[30108]: osdmap e190: 8 total, 8 up, 8 in 2024-03-24T12:14:06.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2179630808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:06.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:06.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[30108]: pgmap v243: 291 pgs: 291 active+clean; 585 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:14:06.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-03-24T12:14:06.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:06.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2179630808' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:06.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-03-24T12:14:06.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:06.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2179630808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi006-81775-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:06.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi006-81775-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:06.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1152523269' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi006-81775-30"}]': finished 2024-03-24T12:14:06.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[41734]: osdmap e190: 8 total, 8 up, 8 in 2024-03-24T12:14:06.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2179630808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:06.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:06.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[41734]: pgmap v243: 291 pgs: 291 active+clean; 585 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:14:06.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-03-24T12:14:06.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:06.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2179630808' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:06.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-03-24T12:14:06.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:06.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2179630808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi006-81775-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:06.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:05 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi006-81775-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:07.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:07 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-03-24T12:14:07.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:07 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi006-81775-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:07.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:07 smithi090 ceph-mon[33646]: osdmap e191: 8 total, 8 up, 8 in 2024-03-24T12:14:07.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:07 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2179630808' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi006-81775-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:07.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:07 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:07.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:07 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/780733571' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi006-81794-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:07.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:07 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi006-81775-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:07.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:07 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:07.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:07 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi006-81794-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:07.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:07 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:07.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:07 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:07.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:07 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-24T12:14:07.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:07 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-24T12:14:07.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-03-24T12:14:07.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi006-81775-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:07.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[30108]: osdmap e191: 8 total, 8 up, 8 in 2024-03-24T12:14:07.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2179630808' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi006-81775-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:07.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:07.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/780733571' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi006-81794-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:07.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi006-81775-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:07.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:07.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi006-81794-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:07.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:07.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:07.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-24T12:14:07.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-24T12:14:07.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-03-24T12:14:07.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi006-81775-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:07.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[41734]: osdmap e191: 8 total, 8 up, 8 in 2024-03-24T12:14:07.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2179630808' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi006-81775-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:07.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:07.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/780733571' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi006-81794-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:07.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi006-81775-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:07.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:07.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi006-81794-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:07.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:07.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:07.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-24T12:14:07.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:07 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-24T12:14:08.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:08.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi006-81794-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:08.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pgp_num","val":"11"}]': finished 2024-03-24T12:14:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:08 smithi090 ceph-mon[33646]: pgmap v246: 355 pgs: 32 unknown, 323 active+clean; 585 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 25 KiB/s rd, 0 B/s wr, 25 op/s 2024-03-24T12:14:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:08 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-03-24T12:14:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:08 smithi090 ceph-mon[33646]: osdmap e192: 8 total, 8 up, 8 in 2024-03-24T12:14:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:08 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-03-24T12:14:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi006-81775-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi006-81775-31"}]': finished 2024-03-24T12:14:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-19", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:14:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:08 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pg_num_actual", "val": "30"}]': finished 2024-03-24T12:14:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:08 smithi090 ceph-mon[33646]: osdmap e193: 8 total, 8 up, 8 in 2024-03-24T12:14:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-19"}]: dispatch 2024-03-24T12:14:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-19"}]: dispatch 2024-03-24T12:14:08.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/257170133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:08.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:08.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/257170133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi006-81794-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pgp_num","val":"11"}]': finished 2024-03-24T12:14:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:08.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:08.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[30108]: pgmap v246: 355 pgs: 32 unknown, 323 active+clean; 585 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 25 KiB/s rd, 0 B/s wr, 25 op/s 2024-03-24T12:14:08.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-03-24T12:14:08.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[30108]: osdmap e192: 8 total, 8 up, 8 in 2024-03-24T12:14:08.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:08.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-03-24T12:14:08.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi006-81775-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi006-81775-31"}]': finished 2024-03-24T12:14:08.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-19", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:14:08.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pg_num_actual", "val": "30"}]': finished 2024-03-24T12:14:08.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[30108]: osdmap e193: 8 total, 8 up, 8 in 2024-03-24T12:14:08.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-19"}]: dispatch 2024-03-24T12:14:08.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-19"}]: dispatch 2024-03-24T12:14:08.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/257170133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:08.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:08.464 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/257170133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:08.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:08.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi006-81794-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:08.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi006-81868-1","var":"pgp_num","val":"11"}]': finished 2024-03-24T12:14:08.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3418082479' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-24T12:14:08.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:08.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[41734]: pgmap v246: 355 pgs: 32 unknown, 323 active+clean; 585 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 25 KiB/s rd, 0 B/s wr, 25 op/s 2024-03-24T12:14:08.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-03-24T12:14:08.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[41734]: osdmap e192: 8 total, 8 up, 8 in 2024-03-24T12:14:08.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:08.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-03-24T12:14:08.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi006-81775-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi006-81775-31"}]': finished 2024-03-24T12:14:08.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-19", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:14:08.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi006-81868-1", "var": "pg_num_actual", "val": "30"}]': finished 2024-03-24T12:14:08.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[41734]: osdmap e193: 8 total, 8 up, 8 in 2024-03-24T12:14:08.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-19"}]: dispatch 2024-03-24T12:14:08.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-19"}]: dispatch 2024-03-24T12:14:08.466 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/257170133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:08.466 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:08.466 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/257170133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:09.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:09 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:09.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:09 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/257170133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi006-81868-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:09.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:09 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi006-81868-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:09.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:09 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-19"}]': finished 2024-03-24T12:14:09.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:09 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi006-81868-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:09.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:09 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-19", "mode": "writeback"}]: dispatch 2024-03-24T12:14:09.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:09 smithi090 ceph-mon[33646]: osdmap e194: 8 total, 8 up, 8 in 2024-03-24T12:14:09.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:09 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/257170133' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi006-81868-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:09.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:09 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4045817518' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi006-81794-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:09.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:09 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi006-81868-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:09.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:09 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-19", "mode": "writeback"}]: dispatch 2024-03-24T12:14:09.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:09 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi006-81794-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/257170133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi006-81868-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi006-81868-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-19"}]': finished 2024-03-24T12:14:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi006-81868-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-19", "mode": "writeback"}]: dispatch 2024-03-24T12:14:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[30108]: osdmap e194: 8 total, 8 up, 8 in 2024-03-24T12:14:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/257170133' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi006-81868-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:09.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4045817518' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi006-81794-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:09.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi006-81868-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:09.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-19", "mode": "writeback"}]: dispatch 2024-03-24T12:14:09.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi006-81794-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:09.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:09.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/257170133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi006-81868-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:09.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi006-81868-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:09.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-19"}]': finished 2024-03-24T12:14:09.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi006-81868-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:09.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-19", "mode": "writeback"}]: dispatch 2024-03-24T12:14:09.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[41734]: osdmap e194: 8 total, 8 up, 8 in 2024-03-24T12:14:09.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/257170133' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi006-81868-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:09.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4045817518' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi006-81794-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:09.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi006-81868-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:09.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-19", "mode": "writeback"}]: dispatch 2024-03-24T12:14:09.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:09 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi006-81794-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:10.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:10 smithi090 ceph-mon[33646]: pgmap v249: 332 pgs: 40 unknown, 292 active+clean; 583 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:14:10.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:10 smithi090 ceph-mon[33646]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:10.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:10 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:14:10.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:10 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-19", "mode": "writeback"}]': finished 2024-03-24T12:14:10.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:10 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi006-81794-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:10.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:10 smithi090 ceph-mon[33646]: osdmap e195: 8 total, 8 up, 8 in 2024-03-24T12:14:10.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2179630808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:10.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:10 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:10.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:10 smithi006 ceph-mon[30108]: pgmap v249: 332 pgs: 40 unknown, 292 active+clean; 583 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:14:10.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:10 smithi006 ceph-mon[30108]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:10.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:10 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:14:10.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:10 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-19", "mode": "writeback"}]': finished 2024-03-24T12:14:10.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:10 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi006-81794-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:10.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:10 smithi006 ceph-mon[30108]: osdmap e195: 8 total, 8 up, 8 in 2024-03-24T12:14:10.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2179630808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:10.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:10 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:10.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:10 smithi006 ceph-mon[41734]: pgmap v249: 332 pgs: 40 unknown, 292 active+clean; 583 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:14:10.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:10 smithi006 ceph-mon[41734]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:10.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:10 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:14:10.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:10 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-19", "mode": "writeback"}]': finished 2024-03-24T12:14:10.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:10 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi006-81794-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:10.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:10 smithi006 ceph-mon[41734]: osdmap e195: 8 total, 8 up, 8 in 2024-03-24T12:14:10.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2179630808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:10.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:10 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:11.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:11.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:11.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi006-81868-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi006-81868-2"}]': finished 2024-03-24T12:14:11.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi006-81775-31"}]': finished 2024-03-24T12:14:11.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:14:11.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:11 smithi090 ceph-mon[33646]: osdmap e196: 8 total, 8 up, 8 in 2024-03-24T12:14:11.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2179630808' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:11.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-19"}]: dispatch 2024-03-24T12:14:11.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:11.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-19"}]: dispatch 2024-03-24T12:14:11.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:11.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:11.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi006-81868-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi006-81868-2"}]': finished 2024-03-24T12:14:11.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi006-81775-31"}]': finished 2024-03-24T12:14:11.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:14:11.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:11 smithi006 ceph-mon[30108]: osdmap e196: 8 total, 8 up, 8 in 2024-03-24T12:14:11.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2179630808' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:11.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-19"}]: dispatch 2024-03-24T12:14:11.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:11.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-19"}]: dispatch 2024-03-24T12:14:11.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:11.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:11.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi006-81868-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi006-81868-2"}]': finished 2024-03-24T12:14:11.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi006-81775-31"}]': finished 2024-03-24T12:14:11.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:14:11.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:11 smithi006 ceph-mon[41734]: osdmap e196: 8 total, 8 up, 8 in 2024-03-24T12:14:11.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2179630808' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:11.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-19"}]: dispatch 2024-03-24T12:14:11.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi006-81775-31"}]: dispatch 2024-03-24T12:14:11.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-19"}]: dispatch 2024-03-24T12:14:11.939 INFO:tasks.workunit.client.0.smithi006.stdout: expected=1 2024-03-24T12:14:11.939 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:e960b815:::9:head 2024-03-24T12:14:11.939 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : cursor()=11:e960b815:::9:head expected=11:e960b815:::9:head 2024-03-24T12:14:11.939 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:e960b815:::9:head -> 9 2024-03-24T12:14:11.939 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : entry=9 expected=9 2024-03-24T12:14:11.939 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:de5d7c5f:::12:head 2024-03-24T12:14:11.939 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : cursor()=11:de5d7c5f:::12:head expected=11:de5d7c5f:::12:head 2024-03-24T12:14:11.939 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:de5d7c5f:::12:head -> 12 2024-03-24T12:14:11.940 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : entry=12 expected=12 2024-03-24T12:14:11.940 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:d83876eb:::4:head 2024-03-24T12:14:11.940 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : cursor()=11:d83876eb:::4:head expected=11:d83876eb:::4:head 2024-03-24T12:14:11.940 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:d83876eb:::4:head -> 4 2024-03-24T12:14:11.940 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : entry=4 expected=4 2024-03-24T12:14:11.940 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:5c6b0b28:::7:head 2024-03-24T12:14:11.940 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : cursor()=11:5c6b0b28:::7:head expected=11:5c6b0b28:::7:head 2024-03-24T12:14:11.940 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:5c6b0b28:::7:head -> 7 2024-03-24T12:14:11.940 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : entry=7 expected=7 2024-03-24T12:14:11.940 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:566253c9:::13:head 2024-03-24T12:14:11.940 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : cursor()=11:566253c9:::13:head expected=11:566253c9:::13:head 2024-03-24T12:14:11.940 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:566253c9:::13:head -> 13 2024-03-24T12:14:11.940 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : entry=13 expected=13 2024-03-24T12:14:11.940 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:863748b0:::15:head 2024-03-24T12:14:11.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : cursor()=11:863748b0:::15:head expected=11:863748b0:::15:head 2024-03-24T12:14:11.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:863748b0:::15:head -> 15 2024-03-24T12:14:11.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : entry=15 expected=15 2024-03-24T12:14:11.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:6cac518f:::0:head 2024-03-24T12:14:11.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : cursor()=11:6cac518f:::0:head expected=11:6cac518f:::0:head 2024-03-24T12:14:11.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:6cac518f:::0:head -> 0 2024-03-24T12:14:11.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : entry=0 expected=0 2024-03-24T12:14:11.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : seek to 11:62a1935d:::14:head 2024-03-24T12:14:11.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : cursor()=11:62a1935d:::14:head expected=11:62a1935d:::14:head 2024-03-24T12:14:11.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: > 11:62a1935d:::14:head -> 14 2024-03-24T12:14:11.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: : entry=14 expected=14 2024-03-24T12:14:11.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (240 ms) 2024-03-24T12:14:11.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-03-24T12:14:11.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (86610 ms) 2024-03-24T12:14:11.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-03-24T12:14:11.941 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: split 0/5 -> MIN 11:33333333::::head 2024-03-24T12:14:11.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: split 1/5 -> 11:33333333::::head 11:66666666::::head 2024-03-24T12:14:11.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: split 2/5 -> 11:66666666::::head 11:99999999::::head 2024-03-24T12:14:11.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: split 3/5 -> 11:99999999::::head 11:cccccccc::::head 2024-03-24T12:14:11.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: split 4/5 -> 11:cccccccc::::head MAX 2024-03-24T12:14:11.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (27804 ms) 2024-03-24T12:14:11.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [----------] 7 tests from LibRadosList (116207 ms total) 2024-03-24T12:14:11.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 2024-03-24T12:14:11.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-03-24T12:14:11.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-03-24T12:14:11.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1041 ms) 2024-03-24T12:14:11.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-03-24T12:14:11.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: myset foo1,foo2,foo3 2024-03-24T12:14:11.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: foo1 2024-03-24T12:14:11.942 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: foo2 2024-03-24T12:14:11.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: foo3 2024-03-24T12:14:11.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: myset foo1,foo4,foo5 2024-03-24T12:14:11.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: foo4 2024-03-24T12:14:11.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: foo5 2024-03-24T12:14:11.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: foo1 2024-03-24T12:14:11.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: myset foo6,foo7 2024-03-24T12:14:11.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: foo7 2024-03-24T12:14:11.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: foo6 2024-03-24T12:14:11.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-03-24T12:14:11.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: ns1:foo4 2024-03-24T12:14:11.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: ns1:foo5 2024-03-24T12:14:11.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: ns2:foo7 2024-03-24T12:14:11.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: ns2:foo6 2024-03-24T12:14:11.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: ns1:foo1 2024-03-24T12:14:11.943 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: :foo1 2024-03-24T12:14:11.944 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: :foo2 2024-03-24T12:14:11.944 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: :foo3 2024-03-24T12:14:11.944 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (32 ms) 2024-03-24T12:14:12.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:14:12 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:14:12.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:12 smithi090 ceph-mon[33646]: pgmap v252: 300 pgs: 8 unknown, 292 active+clean; 583 KiB data, 572 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:14:12.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:12 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:14:12.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi006-81775-31"}]': finished 2024-03-24T12:14:12.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-19"}]': finished 2024-03-24T12:14:12.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:12 smithi090 ceph-mon[33646]: osdmap e197: 8 total, 8 up, 8 in 2024-03-24T12:14:12.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:12 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/783107783' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:12.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:12.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:12 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4043439796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:12.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:12.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:12 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4043439796' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:12.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:12.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:12 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4043439796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi006-81775-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:12.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi006-81775-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:12.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[30108]: pgmap v252: 300 pgs: 8 unknown, 292 active+clean; 583 KiB data, 572 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:14:12.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:14:12.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi006-81775-31"}]': finished 2024-03-24T12:14:12.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-19"}]': finished 2024-03-24T12:14:12.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[30108]: osdmap e197: 8 total, 8 up, 8 in 2024-03-24T12:14:12.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/783107783' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:12.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:12.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4043439796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:12.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:12.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4043439796' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:12.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:12.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4043439796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi006-81775-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:12.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi006-81775-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:12.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[41734]: pgmap v252: 300 pgs: 8 unknown, 292 active+clean; 583 KiB data, 572 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:14:12.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:14:12.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi006-81775-31"}]': finished 2024-03-24T12:14:12.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-19"}]': finished 2024-03-24T12:14:12.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[41734]: osdmap e197: 8 total, 8 up, 8 in 2024-03-24T12:14:12.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/783107783' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:12.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:12.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4043439796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:12.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:12.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4043439796' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:12.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:12.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4043439796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi006-81775-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:12.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi006-81775-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:13.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:13 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:14:13.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:13 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:13.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:13 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi006-81775-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:13.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:13 smithi090 ceph-mon[33646]: osdmap e198: 8 total, 8 up, 8 in 2024-03-24T12:14:13.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:13 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4043439796' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi006-81775-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:13.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:13 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/257170133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:13.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:13 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi006-81775-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:13.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:13 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:13.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:13 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:14:13.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:13 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:13.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:13 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi006-81775-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:13.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:13 smithi006 ceph-mon[30108]: osdmap e198: 8 total, 8 up, 8 in 2024-03-24T12:14:13.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:13 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4043439796' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi006-81775-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:13.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:13 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/257170133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:13.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:13 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi006-81775-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:13.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:13 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:13.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:13 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:14:13.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:13 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:13.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:13 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi006-81775-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:13.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:13 smithi006 ceph-mon[41734]: osdmap e198: 8 total, 8 up, 8 in 2024-03-24T12:14:13.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:13 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4043439796' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi006-81775-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:13.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:13 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/257170133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:13.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:13 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi006-81775-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:13.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:13 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:14.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:14 smithi090 ceph-mon[33646]: pgmap v255: 292 pgs: 32 creating+peering, 260 active+clean; 583 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:14:14.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:14 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi006-81868-2"}]': finished 2024-03-24T12:14:14.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/257170133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:14.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:14 smithi090 ceph-mon[33646]: osdmap e199: 8 total, 8 up, 8 in 2024-03-24T12:14:14.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:14 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:14.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2203194801' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:14.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:14.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:14 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:14.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:14 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:14.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:14 smithi006 ceph-mon[30108]: pgmap v255: 292 pgs: 32 creating+peering, 260 active+clean; 583 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:14:14.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:14 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi006-81868-2"}]': finished 2024-03-24T12:14:14.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/257170133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:14.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:14 smithi006 ceph-mon[30108]: osdmap e199: 8 total, 8 up, 8 in 2024-03-24T12:14:14.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:14 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:14.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2203194801' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:14.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:14.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:14 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:14.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:14 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:14.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:14 smithi006 ceph-mon[41734]: pgmap v255: 292 pgs: 32 creating+peering, 260 active+clean; 583 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:14:14.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:14 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi006-81868-2"}]': finished 2024-03-24T12:14:14.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/257170133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:14.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:14 smithi006 ceph-mon[41734]: osdmap e199: 8 total, 8 up, 8 in 2024-03-24T12:14:14.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:14 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi006-81868-2"}]: dispatch 2024-03-24T12:14:14.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2203194801' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:14.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:14.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:14 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:14.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:14 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:15.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:14:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:14:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:14:16.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:15 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi006-81775-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi006-81775-32"}]': finished 2024-03-24T12:14:16.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:15 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi006-81868-2"}]': finished 2024-03-24T12:14:16.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:15 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:15 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:15 smithi090 ceph-mon[33646]: osdmap e200: 8 total, 8 up, 8 in 2024-03-24T12:14:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:15 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:15 smithi090 ceph-mon[33646]: pgmap v258: 364 pgs: 72 unknown, 32 creating+peering, 260 active+clean; 583 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:14:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:15 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:16.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:15 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi006-81775-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi006-81775-32"}]': finished 2024-03-24T12:14:16.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:15 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi006-81868-2"}]': finished 2024-03-24T12:14:16.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:15 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:16.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:15 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:16.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:15 smithi006 ceph-mon[30108]: osdmap e200: 8 total, 8 up, 8 in 2024-03-24T12:14:16.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:16.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:15 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:16.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:15 smithi006 ceph-mon[30108]: pgmap v258: 364 pgs: 72 unknown, 32 creating+peering, 260 active+clean; 583 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:14:16.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:15 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:16.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:15 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi006-81775-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi006-81775-32"}]': finished 2024-03-24T12:14:16.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:15 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi006-81868-2"}]': finished 2024-03-24T12:14:16.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:15 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:16.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:15 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:16.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:15 smithi006 ceph-mon[41734]: osdmap e200: 8 total, 8 up, 8 in 2024-03-24T12:14:16.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:16.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:15 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:16.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:15 smithi006 ceph-mon[41734]: pgmap v258: 364 pgs: 72 unknown, 32 creating+peering, 260 active+clean; 583 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:14:16.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:15 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:17.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:16 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-21", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:14:17.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:16 smithi090 ceph-mon[33646]: osdmap e201: 8 total, 8 up, 8 in 2024-03-24T12:14:17.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:16 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-21"}]: dispatch 2024-03-24T12:14:17.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:16 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4037215398' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-81868-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:17.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:16 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-21"}]: dispatch 2024-03-24T12:14:17.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:16 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-81868-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:17.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:16 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3778813321' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:17.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:16 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:17.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:16 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-21", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:14:17.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:16 smithi006 ceph-mon[30108]: osdmap e201: 8 total, 8 up, 8 in 2024-03-24T12:14:17.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:16 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-21"}]: dispatch 2024-03-24T12:14:17.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:16 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4037215398' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-81868-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:17.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:16 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-21"}]: dispatch 2024-03-24T12:14:17.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:16 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-81868-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:17.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:16 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3778813321' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:17.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:16 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:17.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:16 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-21", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:14:17.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:16 smithi006 ceph-mon[41734]: osdmap e201: 8 total, 8 up, 8 in 2024-03-24T12:14:17.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:16 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-21"}]: dispatch 2024-03-24T12:14:17.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:16 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4037215398' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-81868-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:17.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:16 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-21"}]: dispatch 2024-03-24T12:14:17.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:16 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-81868-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:17.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:16 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3778813321' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:17.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:16 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:17.914 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2024-03-24T12:14:17.914 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 1 0 2024-03-24T12:14:17.914 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 10 0 2024-03-24T12:14:17.914 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 13 0 2024-03-24T12:14:17.914 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 7 0 2024-03-24T12:14:17.914 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 14 0 2024-03-24T12:14:17.914 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 0 0 2024-03-24T12:14:17.914 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 15 0 2024-03-24T12:14:17.914 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 11 0 2024-03-24T12:14:17.915 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 5 0 2024-03-24T12:14:17.915 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 8 0 2024-03-24T12:14:17.915 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 6 0 2024-03-24T12:14:17.915 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 3 0 2024-03-24T12:14:17.915 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 4 0 2024-03-24T12:14:17.915 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 12 0 2024-03-24T12:14:17.915 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 9 0 2024-03-24T12:14:17.915 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 2 0 2024-03-24T12:14:17.915 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-03-24T12:14:17.915 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (44 ms) 2024-03-24T12:14:17.915 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [----------] 3 tests from LibRadosListEC (1117 ms total) 2024-03-24T12:14:17.915 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 2024-03-24T12:14:17.915 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-03-24T12:14:17.915 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-03-24T12:14:17.916 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3023 ms) 2024-03-24T12:14:17.916 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [----------] 1 test from LibRadosListNP (3023 ms total) 2024-03-24T12:14:17.916 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: 2024-03-24T12:14:17.916 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [----------] Global test environment tear-down 2024-03-24T12:14:17.916 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [==========] 11 tests from 3 test suites ran. (136505 ms total) 2024-03-24T12:14:17.916 INFO:tasks.workunit.client.0.smithi006.stdout: api_list: [ PASSED ] 11 tests. 2024-03-24T12:14:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-21"}]': finished 2024-03-24T12:14:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-81868-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:17 smithi090 ceph-mon[33646]: osdmap e202: 8 total, 8 up, 8 in 2024-03-24T12:14:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-21", "mode": "writeback"}]: dispatch 2024-03-24T12:14:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4043439796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:18.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-21", "mode": "writeback"}]: dispatch 2024-03-24T12:14:18.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:18.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4037215398' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi006-81868-3","pool2":"test-rados-api-smithi006-81868-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-24T12:14:18.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi006-81868-3","pool2":"test-rados-api-smithi006-81868-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-24T12:14:18.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:17 smithi090 ceph-mon[33646]: pgmap v261: 420 pgs: 64 unknown, 17 creating+peering, 339 active+clean; 584 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 255 B/s wr, 0 op/s 2024-03-24T12:14:18.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:17 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:14:18.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:17 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:14:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-21"}]': finished 2024-03-24T12:14:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-81868-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[30108]: osdmap e202: 8 total, 8 up, 8 in 2024-03-24T12:14:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-21", "mode": "writeback"}]: dispatch 2024-03-24T12:14:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4043439796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-21", "mode": "writeback"}]: dispatch 2024-03-24T12:14:18.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:18.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4037215398' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi006-81868-3","pool2":"test-rados-api-smithi006-81868-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-24T12:14:18.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi006-81868-3","pool2":"test-rados-api-smithi006-81868-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-24T12:14:18.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[30108]: pgmap v261: 420 pgs: 64 unknown, 17 creating+peering, 339 active+clean; 584 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 255 B/s wr, 0 op/s 2024-03-24T12:14:18.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:14:18.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:14:18.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-21"}]': finished 2024-03-24T12:14:18.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-81868-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:18.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi006-81794-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:18.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[41734]: osdmap e202: 8 total, 8 up, 8 in 2024-03-24T12:14:18.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-21", "mode": "writeback"}]: dispatch 2024-03-24T12:14:18.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4043439796' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:18.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-21", "mode": "writeback"}]: dispatch 2024-03-24T12:14:18.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:18.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4037215398' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi006-81868-3","pool2":"test-rados-api-smithi006-81868-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-24T12:14:18.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi006-81868-3","pool2":"test-rados-api-smithi006-81868-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-24T12:14:18.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[41734]: pgmap v261: 420 pgs: 64 unknown, 17 creating+peering, 339 active+clean; 584 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 255 B/s wr, 0 op/s 2024-03-24T12:14:18.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:14:18.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:17 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:14:19.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:18 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:14:19.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-21", "mode": "writeback"}]': finished 2024-03-24T12:14:19.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi006-81775-32"}]': finished 2024-03-24T12:14:19.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi006-81868-3","pool2":"test-rados-api-smithi006-81868-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-03-24T12:14:19.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:18 smithi090 ceph-mon[33646]: osdmap e203: 8 total, 8 up, 8 in 2024-03-24T12:14:19.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4043439796' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:19.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:19.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:19.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:19.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:18 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:14:19.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-21", "mode": "writeback"}]': finished 2024-03-24T12:14:19.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi006-81775-32"}]': finished 2024-03-24T12:14:19.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi006-81868-3","pool2":"test-rados-api-smithi006-81868-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-03-24T12:14:19.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:18 smithi006 ceph-mon[30108]: osdmap e203: 8 total, 8 up, 8 in 2024-03-24T12:14:19.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4043439796' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:19.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:19.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:19.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:19.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:18 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:14:19.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-21", "mode": "writeback"}]': finished 2024-03-24T12:14:19.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi006-81775-32"}]': finished 2024-03-24T12:14:19.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi006-81868-3","pool2":"test-rados-api-smithi006-81868-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-03-24T12:14:19.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:18 smithi006 ceph-mon[41734]: osdmap e203: 8 total, 8 up, 8 in 2024-03-24T12:14:19.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4043439796' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:19.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi006-81775-32"}]: dispatch 2024-03-24T12:14:19.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:19.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:20.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi006-81775-32"}]': finished 2024-03-24T12:14:20.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:14:20.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:19 smithi090 ceph-mon[33646]: osdmap e204: 8 total, 8 up, 8 in 2024-03-24T12:14:20.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-21"}]: dispatch 2024-03-24T12:14:20.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-21"}]: dispatch 2024-03-24T12:14:20.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3083453273' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:20.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:20.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3083453273' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:20.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:20.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3083453273' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi006-81775-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:20.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi006-81775-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:20.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:19 smithi090 ceph-mon[33646]: pgmap v264: 324 pgs: 324 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:14:20.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi006-81775-32"}]': finished 2024-03-24T12:14:20.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:14:20.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[41734]: osdmap e204: 8 total, 8 up, 8 in 2024-03-24T12:14:20.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-21"}]: dispatch 2024-03-24T12:14:20.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-21"}]: dispatch 2024-03-24T12:14:20.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3083453273' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:20.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:20.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3083453273' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:20.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:20.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3083453273' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi006-81775-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:20.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi006-81775-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:20.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[41734]: pgmap v264: 324 pgs: 324 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:14:20.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi006-81775-32"}]': finished 2024-03-24T12:14:20.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:14:20.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[30108]: osdmap e204: 8 total, 8 up, 8 in 2024-03-24T12:14:20.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-21"}]: dispatch 2024-03-24T12:14:20.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-21"}]: dispatch 2024-03-24T12:14:20.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3083453273' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:20.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:20.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3083453273' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:20.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:20.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3083453273' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi006-81775-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:20.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi006-81775-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:20.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:19 smithi006 ceph-mon[30108]: pgmap v264: 324 pgs: 324 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:14:21.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:20 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:14:21.182 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:20 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-21"}]': finished 2024-03-24T12:14:21.182 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:20 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi006-81775-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:21.182 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:20 smithi090 ceph-mon[33646]: osdmap e205: 8 total, 8 up, 8 in 2024-03-24T12:14:21.182 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:20 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3083453273' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi006-81775-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:21.182 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:20 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi006-81775-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:21.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:20 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:14:21.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:20 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-21"}]': finished 2024-03-24T12:14:21.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:20 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi006-81775-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:21.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:20 smithi006 ceph-mon[30108]: osdmap e205: 8 total, 8 up, 8 in 2024-03-24T12:14:21.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:20 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3083453273' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi006-81775-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:21.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:20 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi006-81775-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:21.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:20 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:14:21.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:20 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-21"}]': finished 2024-03-24T12:14:21.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:20 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi006-81775-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:21.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:20 smithi006 ceph-mon[41734]: osdmap e205: 8 total, 8 up, 8 in 2024-03-24T12:14:21.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:20 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3083453273' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi006-81775-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:21.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:20 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi006-81775-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:22.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:21 smithi006 ceph-mon[30108]: pgmap v267: 292 pgs: 32 unknown, 260 active+clean; 583 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:14:22.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:21 smithi006 ceph-mon[30108]: osdmap e206: 8 total, 8 up, 8 in 2024-03-24T12:14:22.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/644750614' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:22.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:21 smithi006 ceph-mon[41734]: pgmap v267: 292 pgs: 32 unknown, 260 active+clean; 583 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:14:22.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:21 smithi006 ceph-mon[41734]: osdmap e206: 8 total, 8 up, 8 in 2024-03-24T12:14:22.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/644750614' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:22.244 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:21 smithi090 ceph-mon[33646]: pgmap v267: 292 pgs: 32 unknown, 260 active+clean; 583 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:14:22.244 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:21 smithi090 ceph-mon[33646]: osdmap e206: 8 total, 8 up, 8 in 2024-03-24T12:14:22.244 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/644750614' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:22.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:14:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:14:23.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:22 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:23.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi006-81775-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi006-81775-33"}]': finished 2024-03-24T12:14:23.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/644750614' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:23.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:22 smithi006 ceph-mon[30108]: osdmap e207: 8 total, 8 up, 8 in 2024-03-24T12:14:23.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:23.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:23.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:22 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:14:23.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:22 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:23.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi006-81775-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi006-81775-33"}]': finished 2024-03-24T12:14:23.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/644750614' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:23.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:22 smithi006 ceph-mon[41734]: osdmap e207: 8 total, 8 up, 8 in 2024-03-24T12:14:23.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:23.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:23.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:22 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:14:23.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:22 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:23.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi006-81775-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi006-81775-33"}]': finished 2024-03-24T12:14:23.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/644750614' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:23.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:22 smithi090 ceph-mon[33646]: osdmap e207: 8 total, 8 up, 8 in 2024-03-24T12:14:23.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:23.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:23.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:22 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:14:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:23 smithi006 ceph-mon[30108]: osdmap e208: 8 total, 8 up, 8 in 2024-03-24T12:14:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3263485307' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:23 smithi006 ceph-mon[30108]: pgmap v270: 364 pgs: 32 unknown, 40 creating+peering, 292 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:14:24.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:24.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:23 smithi006 ceph-mon[41734]: osdmap e208: 8 total, 8 up, 8 in 2024-03-24T12:14:24.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3263485307' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:24.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:23 smithi006 ceph-mon[41734]: pgmap v270: 364 pgs: 32 unknown, 40 creating+peering, 292 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:14:24.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:24.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:23 smithi090 ceph-mon[33646]: osdmap e208: 8 total, 8 up, 8 in 2024-03-24T12:14:24.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3263485307' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:24.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:23 smithi090 ceph-mon[33646]: pgmap v270: 364 pgs: 32 unknown, 40 creating+peering, 292 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:14:25.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3263485307' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:25.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:24 smithi006 ceph-mon[30108]: osdmap e209: 8 total, 8 up, 8 in 2024-03-24T12:14:25.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3083453273' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:25.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:25.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:25.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:25.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3263485307' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:25.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:24 smithi006 ceph-mon[41734]: osdmap e209: 8 total, 8 up, 8 in 2024-03-24T12:14:25.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3083453273' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:25.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:25.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:25.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:25.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3263485307' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:25.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:24 smithi090 ceph-mon[33646]: osdmap e209: 8 total, 8 up, 8 in 2024-03-24T12:14:25.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3083453273' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:25.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:25.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:25.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:25.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:14:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:14:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:14:26.330 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:25 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi006-81775-33"}]': finished 2024-03-24T12:14:26.330 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:25 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-23", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:14:26.330 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:25 smithi090 ceph-mon[33646]: osdmap e210: 8 total, 8 up, 8 in 2024-03-24T12:14:26.330 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-23"}]: dispatch 2024-03-24T12:14:26.330 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:25 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-23"}]: dispatch 2024-03-24T12:14:26.330 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3083453273' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:26.330 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2160040171' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:26.330 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:25 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:26.330 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:25 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:26.330 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:25 smithi090 ceph-mon[33646]: pgmap v273: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:14:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:25 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi006-81775-33"}]': finished 2024-03-24T12:14:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:25 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-23", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:14:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:25 smithi006 ceph-mon[30108]: osdmap e210: 8 total, 8 up, 8 in 2024-03-24T12:14:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-23"}]: dispatch 2024-03-24T12:14:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:25 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-23"}]: dispatch 2024-03-24T12:14:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3083453273' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2160040171' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:25 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:25 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:26.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:25 smithi006 ceph-mon[30108]: pgmap v273: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:14:26.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:25 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi006-81775-33"}]': finished 2024-03-24T12:14:26.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:25 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-23", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:14:26.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:25 smithi006 ceph-mon[41734]: osdmap e210: 8 total, 8 up, 8 in 2024-03-24T12:14:26.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-23"}]: dispatch 2024-03-24T12:14:26.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:25 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-23"}]: dispatch 2024-03-24T12:14:26.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3083453273' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:26.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2160040171' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:26.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:25 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi006-81775-33"}]: dispatch 2024-03-24T12:14:26.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:25 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:26.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:25 smithi006 ceph-mon[41734]: pgmap v273: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:14:26.476 INFO:tasks.workunit.client.0.smithi006.stdout: 0x55ac4ada8680 2024-03-24T12:14:26.477 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada7d60 2024-03-24T12:14:26.477 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada7eb0 2024-03-24T12:14:26.477 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada5490 2024-03-24T12:14:26.477 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada55c0 2024-03-24T12:14:26.477 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada5710 2024-03-24T12:14:26.477 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada7fc0 2024-03-24T12:14:26.477 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada8c50 2024-03-24T12:14:26.477 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada87b0 2024-03-24T12:14:26.477 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada88c0 2024-03-24T12:14:26.477 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada8aa0 2024-03-24T12:14:26.478 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada8d80 2024-03-24T12:14:26.478 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada8ed0 2024-03-24T12:14:26.478 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada9480 2024-03-24T12:14:26.478 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada8fe0 2024-03-24T12:14:26.478 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada9110 2024-03-24T12:14:26.478 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada9220 2024-03-24T12:14:26.478 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada9330 2024-03-24T12:14:26.478 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada9a30 2024-03-24T12:14:26.478 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada95e0 2024-03-24T12:14:26.478 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada9730 2024-03-24T12:14:26.478 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada9860 2024-03-24T12:14:26.478 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada9b60 2024-03-24T12:14:26.478 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada9c70 2024-03-24T12:14:26.478 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaa240 2024-03-24T12:14:26.479 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada9da0 2024-03-24T12:14:26.479 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada9ed0 2024-03-24T12:14:26.479 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaa020 2024-03-24T12:14:26.479 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaa350 2024-03-24T12:14:26.479 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaa7c0 2024-03-24T12:14:26.479 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaa8d0 2024-03-24T12:14:26.479 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada81f0 2024-03-24T12:14:26.479 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada8300 2024-03-24T12:14:26.479 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4ada8430 2024-03-24T12:14:26.479 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaaa20 2024-03-24T12:14:26.479 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaae90 2024-03-24T12:14:26.479 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaab30 2024-03-24T12:14:26.479 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaac40 2024-03-24T12:14:26.479 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaad50 2024-03-24T12:14:26.480 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adab480 2024-03-24T12:14:26.480 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaafe0 2024-03-24T12:14:26.480 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adab0f0 2024-03-24T12:14:26.480 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adab200 2024-03-24T12:14:26.480 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adab350 2024-03-24T12:14:26.480 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaba30 2024-03-24T12:14:26.480 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adab590 2024-03-24T12:14:26.480 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adab6a0 2024-03-24T12:14:26.480 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adab7d0 2024-03-24T12:14:26.480 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adab900 2024-03-24T12:14:26.480 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adac020 2024-03-24T12:14:26.480 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adabb80 2024-03-24T12:14:26.480 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adabc90 2024-03-24T12:14:26.480 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adabda0 2024-03-24T12:14:26.480 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adabed0 2024-03-24T12:14:26.481 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adac5f0 2024-03-24T12:14:26.481 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adac150 2024-03-24T12:14:26.481 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adac260 2024-03-24T12:14:26.481 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adac370 2024-03-24T12:14:26.481 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adac4a0 2024-03-24T12:14:26.481 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adacbc0 2024-03-24T12:14:26.481 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adac720 2024-03-24T12:14:26.481 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adac830 2024-03-24T12:14:26.481 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adac960 2024-03-24T12:14:26.481 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaca90 2024-03-24T12:14:26.481 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adad170 2024-03-24T12:14:26.481 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaccd0 2024-03-24T12:14:26.481 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adacde0 2024-03-24T12:14:26.481 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adacf10 2024-03-24T12:14:26.482 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adad020 2024-03-24T12:14:26.482 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adad760 2024-03-24T12:14:26.482 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adad2c0 2024-03-24T12:14:26.482 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adad3d0 2024-03-24T12:14:26.482 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adad500 2024-03-24T12:14:26.482 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adad630 2024-03-24T12:14:26.482 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adadd60 2024-03-24T12:14:26.482 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adad8c0 2024-03-24T12:14:26.482 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adad9d0 2024-03-24T12:14:26.482 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adadb00 2024-03-24T12:14:26.482 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adade90 2024-03-24T12:14:26.482 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adae340 2024-03-24T12:14:26.482 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adae450 2024-03-24T12:14:26.482 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adadfc0 2024-03-24T12:14:27.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-23"}]': finished 2024-03-24T12:14:27.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi006-81775-33"}]': finished 2024-03-24T12:14:27.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:27.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:26 smithi090 ceph-mon[33646]: osdmap e211: 8 total, 8 up, 8 in 2024-03-24T12:14:27.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-23", "mode": "writeback"}]: dispatch 2024-03-24T12:14:27.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-23", "mode": "writeback"}]: dispatch 2024-03-24T12:14:27.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3118960673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:27.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:27.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3118960673' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:27.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:27.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3118960673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi006-81775-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:27.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi006-81775-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:27.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-23"}]': finished 2024-03-24T12:14:27.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi006-81775-33"}]': finished 2024-03-24T12:14:27.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:27.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[30108]: osdmap e211: 8 total, 8 up, 8 in 2024-03-24T12:14:27.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-23", "mode": "writeback"}]: dispatch 2024-03-24T12:14:27.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-23", "mode": "writeback"}]: dispatch 2024-03-24T12:14:27.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3118960673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:27.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:27.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3118960673' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:27.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:27.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3118960673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi006-81775-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:27.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi006-81775-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:27.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-23"}]': finished 2024-03-24T12:14:27.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi006-81775-33"}]': finished 2024-03-24T12:14:27.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:27.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[41734]: osdmap e211: 8 total, 8 up, 8 in 2024-03-24T12:14:27.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-23", "mode": "writeback"}]: dispatch 2024-03-24T12:14:27.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-23", "mode": "writeback"}]: dispatch 2024-03-24T12:14:27.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3118960673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:27.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:27.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3118960673' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:27.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:27.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3118960673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi006-81775-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:27.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi006-81775-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:28.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:28 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:14:28.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-23", "mode": "writeback"}]': finished 2024-03-24T12:14:28.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi006-81775-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:28.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:28 smithi090 ceph-mon[33646]: osdmap e212: 8 total, 8 up, 8 in 2024-03-24T12:14:28.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3118960673' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi006-81775-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:28.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1600809117' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:28.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi006-81775-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:28.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:28.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:28 smithi090 ceph-mon[33646]: pgmap v276: 420 pgs: 44 unknown, 25 creating+peering, 351 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-24T12:14:28.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:28.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:28 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:14:28.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-23", "mode": "writeback"}]': finished 2024-03-24T12:14:28.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi006-81775-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:28.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:28 smithi006 ceph-mon[30108]: osdmap e212: 8 total, 8 up, 8 in 2024-03-24T12:14:28.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3118960673' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi006-81775-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:28.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1600809117' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:28.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi006-81775-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:28.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:28.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:28 smithi006 ceph-mon[30108]: pgmap v276: 420 pgs: 44 unknown, 25 creating+peering, 351 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-24T12:14:28.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:28.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:28 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:14:28.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-23", "mode": "writeback"}]': finished 2024-03-24T12:14:28.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi006-81775-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:28.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:28 smithi006 ceph-mon[41734]: osdmap e212: 8 total, 8 up, 8 in 2024-03-24T12:14:28.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3118960673' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi006-81775-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:28.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1600809117' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:28.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi006-81775-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:28.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:28.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:28 smithi006 ceph-mon[41734]: pgmap v276: 420 pgs: 44 unknown, 25 creating+peering, 351 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-24T12:14:28.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:29.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:29 smithi090 ceph-mon[33646]: osdmap e213: 8 total, 8 up, 8 in 2024-03-24T12:14:29.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:29 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:29.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:29.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi006-81775-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi006-81775-34"}]': finished 2024-03-24T12:14:29.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:14:29.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:29 smithi090 ceph-mon[33646]: osdmap e214: 8 total, 8 up, 8 in 2024-03-24T12:14:29.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:29 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-23"}]: dispatch 2024-03-24T12:14:29.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-23"}]: dispatch 2024-03-24T12:14:29.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:29 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2777907440' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:29.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:29 smithi006 ceph-mon[30108]: osdmap e213: 8 total, 8 up, 8 in 2024-03-24T12:14:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:29 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi006-81775-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi006-81775-34"}]': finished 2024-03-24T12:14:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:14:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:29 smithi006 ceph-mon[30108]: osdmap e214: 8 total, 8 up, 8 in 2024-03-24T12:14:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:29 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-23"}]: dispatch 2024-03-24T12:14:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-23"}]: dispatch 2024-03-24T12:14:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:29 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2777907440' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:29.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:29.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:29 smithi006 ceph-mon[41734]: osdmap e213: 8 total, 8 up, 8 in 2024-03-24T12:14:29.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:29 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:29.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:29.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi006-81775-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi006-81775-34"}]': finished 2024-03-24T12:14:29.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:14:29.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:29 smithi006 ceph-mon[41734]: osdmap e214: 8 total, 8 up, 8 in 2024-03-24T12:14:29.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:29 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-23"}]: dispatch 2024-03-24T12:14:29.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-23"}]: dispatch 2024-03-24T12:14:29.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:29 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2777907440' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:29.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:30 smithi090 ceph-mon[33646]: pgmap v279: 460 pgs: 40 unknown, 32 creating+peering, 388 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-03-24T12:14:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:30 smithi090 ceph-mon[33646]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:30 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:14:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:30 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-23"}]': finished 2024-03-24T12:14:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:30 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:30.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:30 smithi090 ceph-mon[33646]: osdmap e215: 8 total, 8 up, 8 in 2024-03-24T12:14:30.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:30 smithi006 ceph-mon[30108]: pgmap v279: 460 pgs: 40 unknown, 32 creating+peering, 388 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-03-24T12:14:30.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:30 smithi006 ceph-mon[30108]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:30.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:30 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:14:30.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:30 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-23"}]': finished 2024-03-24T12:14:30.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:30 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:30.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:30 smithi006 ceph-mon[30108]: osdmap e215: 8 total, 8 up, 8 in 2024-03-24T12:14:30.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:30 smithi006 ceph-mon[41734]: pgmap v279: 460 pgs: 40 unknown, 32 creating+peering, 388 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-03-24T12:14:30.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:30 smithi006 ceph-mon[41734]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:30.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:30 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:14:30.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:30 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-23"}]': finished 2024-03-24T12:14:30.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:30 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi006-81794-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:30.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:30 smithi006 ceph-mon[41734]: osdmap e215: 8 total, 8 up, 8 in 2024-03-24T12:14:32.255 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:31 smithi090 ceph-mon[33646]: osdmap e216: 8 total, 8 up, 8 in 2024-03-24T12:14:32.255 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3118960673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:32.255 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:31 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:32.255 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:31 smithi090 ceph-mon[33646]: pgmap v282: 388 pgs: 32 creating+peering, 356 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1023 B/s wr, 0 op/s 2024-03-24T12:14:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:31 smithi006 ceph-mon[30108]: osdmap e216: 8 total, 8 up, 8 in 2024-03-24T12:14:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3118960673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:31 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:31 smithi006 ceph-mon[30108]: pgmap v282: 388 pgs: 32 creating+peering, 356 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1023 B/s wr, 0 op/s 2024-03-24T12:14:32.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:31 smithi006 ceph-mon[41734]: osdmap e216: 8 total, 8 up, 8 in 2024-03-24T12:14:32.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3118960673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:32.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:31 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:32.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:31 smithi006 ceph-mon[41734]: pgmap v282: 388 pgs: 32 creating+peering, 356 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1023 B/s wr, 0 op/s 2024-03-24T12:14:32.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:14:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:14:33.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:33 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi006-81775-34"}]': finished 2024-03-24T12:14:33.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:33 smithi090 ceph-mon[33646]: osdmap e217: 8 total, 8 up, 8 in 2024-03-24T12:14:33.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3118960673' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:33.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:33.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:33 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:33.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:33 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:33.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:33 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:14:33.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:33 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:14:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:33 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi006-81775-34"}]': finished 2024-03-24T12:14:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:33 smithi006 ceph-mon[30108]: osdmap e217: 8 total, 8 up, 8 in 2024-03-24T12:14:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3118960673' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:33 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:33 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:33.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:33 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:14:33.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:33 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:14:33.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:33 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi006-81775-34"}]': finished 2024-03-24T12:14:33.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:33 smithi006 ceph-mon[41734]: osdmap e217: 8 total, 8 up, 8 in 2024-03-24T12:14:33.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3118960673' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:33.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:33.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:33 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi006-81775-34"}]: dispatch 2024-03-24T12:14:33.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:33 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:33.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:33 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:14:33.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:33 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:14:34.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:34 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi006-81775-34"}]': finished 2024-03-24T12:14:34.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:34 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:34.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:34 smithi090 ceph-mon[33646]: osdmap e218: 8 total, 8 up, 8 in 2024-03-24T12:14:34.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3675363937' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:34.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:34 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:34.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3675363937' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:34.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:34 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:34.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3675363937' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi006-81775-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:34.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:34 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi006-81775-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:34.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:34 smithi090 ceph-mon[33646]: pgmap v285: 356 pgs: 32 creating+peering, 324 active+clean; 584 KiB data, 555 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:14:34.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:34 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi006-81775-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:34.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:34 smithi090 ceph-mon[33646]: osdmap e219: 8 total, 8 up, 8 in 2024-03-24T12:14:34.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3675363937' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi006-81775-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:34.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:34 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi006-81775-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:34.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi006-81775-34"}]': finished 2024-03-24T12:14:34.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:34.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[30108]: osdmap e218: 8 total, 8 up, 8 in 2024-03-24T12:14:34.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3675363937' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:34.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:34.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3675363937' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:34.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:34.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3675363937' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi006-81775-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:34.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi006-81775-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:34.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[30108]: pgmap v285: 356 pgs: 32 creating+peering, 324 active+clean; 584 KiB data, 555 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:14:34.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi006-81775-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:34.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[30108]: osdmap e219: 8 total, 8 up, 8 in 2024-03-24T12:14:34.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3675363937' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi006-81775-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:34.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi006-81775-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:34.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi006-81775-34"}]': finished 2024-03-24T12:14:34.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:34.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[41734]: osdmap e218: 8 total, 8 up, 8 in 2024-03-24T12:14:34.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3675363937' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:34.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:34.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3675363937' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:34.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:34.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3675363937' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi006-81775-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:34.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi006-81775-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:34.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[41734]: pgmap v285: 356 pgs: 32 creating+peering, 324 active+clean; 584 KiB data, 555 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:14:34.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi006-81775-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:34.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[41734]: osdmap e219: 8 total, 8 up, 8 in 2024-03-24T12:14:34.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3675363937' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi006-81775-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:34.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:34 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi006-81775-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:35.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:35 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:35.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:35.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-25", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:14:35.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:14:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:14:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:14:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:35 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-25", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:14:35.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:35 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:35.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:35.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-25", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:14:36.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:36 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-25"}]: dispatch 2024-03-24T12:14:36.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:36 smithi090 ceph-mon[33646]: pgmap v288: 292 pgs: 32 creating+peering, 260 active+clean; 583 KiB data, 555 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:14:36.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:36 smithi090 ceph-mon[33646]: osdmap e220: 8 total, 8 up, 8 in 2024-03-24T12:14:36.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:36 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-25"}]: dispatch 2024-03-24T12:14:36.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:36 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:36.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:36 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi006-81775-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi006-81775-35"}]': finished 2024-03-24T12:14:36.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:36 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-25"}]': finished 2024-03-24T12:14:36.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:36 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-25", "mode": "writeback"}]: dispatch 2024-03-24T12:14:36.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:36 smithi090 ceph-mon[33646]: osdmap e221: 8 total, 8 up, 8 in 2024-03-24T12:14:36.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:36 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-25", "mode": "writeback"}]: dispatch 2024-03-24T12:14:36.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:36 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3632940719' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi006-81794-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:36.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:36 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi006-81794-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:36.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-25"}]: dispatch 2024-03-24T12:14:36.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[41734]: pgmap v288: 292 pgs: 32 creating+peering, 260 active+clean; 583 KiB data, 555 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:14:36.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[41734]: osdmap e220: 8 total, 8 up, 8 in 2024-03-24T12:14:36.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-25"}]: dispatch 2024-03-24T12:14:36.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:36.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi006-81775-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi006-81775-35"}]': finished 2024-03-24T12:14:36.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-25"}]': finished 2024-03-24T12:14:36.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-25", "mode": "writeback"}]: dispatch 2024-03-24T12:14:36.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[41734]: osdmap e221: 8 total, 8 up, 8 in 2024-03-24T12:14:36.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-25", "mode": "writeback"}]: dispatch 2024-03-24T12:14:36.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3632940719' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi006-81794-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:36.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi006-81794-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:36.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-25"}]: dispatch 2024-03-24T12:14:36.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[30108]: pgmap v288: 292 pgs: 32 creating+peering, 260 active+clean; 583 KiB data, 555 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:14:36.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[30108]: osdmap e220: 8 total, 8 up, 8 in 2024-03-24T12:14:36.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-25"}]: dispatch 2024-03-24T12:14:36.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:36.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi006-81775-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi006-81775-35"}]': finished 2024-03-24T12:14:36.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-25"}]': finished 2024-03-24T12:14:36.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-25", "mode": "writeback"}]: dispatch 2024-03-24T12:14:36.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[30108]: osdmap e221: 8 total, 8 up, 8 in 2024-03-24T12:14:36.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-25", "mode": "writeback"}]: dispatch 2024-03-24T12:14:36.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3632940719' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi006-81794-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:36.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:36 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi006-81794-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:37.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:37 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:14:37.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:37 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-25", "mode": "writeback"}]': finished 2024-03-24T12:14:37.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:37 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi006-81794-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:37.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:37 smithi090 ceph-mon[33646]: osdmap e222: 8 total, 8 up, 8 in 2024-03-24T12:14:37.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:37.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:37 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:37.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:37 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:14:37.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:37 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-25", "mode": "writeback"}]': finished 2024-03-24T12:14:37.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:37 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi006-81794-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:37.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:37 smithi006 ceph-mon[30108]: osdmap e222: 8 total, 8 up, 8 in 2024-03-24T12:14:37.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:37.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:37 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:37.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:37 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:14:37.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:37 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-25", "mode": "writeback"}]': finished 2024-03-24T12:14:37.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:37 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi006-81794-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:37.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:37 smithi006 ceph-mon[41734]: osdmap e222: 8 total, 8 up, 8 in 2024-03-24T12:14:37.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:37.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:37 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:38.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:38 smithi090 ceph-mon[33646]: pgmap v291: 332 pgs: 21 unknown, 15 creating+peering, 296 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-24T12:14:38.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:14:38.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:38 smithi090 ceph-mon[33646]: osdmap e223: 8 total, 8 up, 8 in 2024-03-24T12:14:38.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:38 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3675363937' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:38.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:38.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:38 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-25"}]: dispatch 2024-03-24T12:14:38.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-25"}]: dispatch 2024-03-24T12:14:38.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:38 smithi006 ceph-mon[30108]: pgmap v291: 332 pgs: 21 unknown, 15 creating+peering, 296 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-24T12:14:38.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:14:38.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:38 smithi006 ceph-mon[30108]: osdmap e223: 8 total, 8 up, 8 in 2024-03-24T12:14:38.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:38 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3675363937' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:38.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:38.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:38 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-25"}]: dispatch 2024-03-24T12:14:38.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-25"}]: dispatch 2024-03-24T12:14:38.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:38 smithi006 ceph-mon[41734]: pgmap v291: 332 pgs: 21 unknown, 15 creating+peering, 296 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-24T12:14:38.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:14:38.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:38 smithi006 ceph-mon[41734]: osdmap e223: 8 total, 8 up, 8 in 2024-03-24T12:14:38.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:38 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3675363937' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:38.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:38.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:38 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-25"}]: dispatch 2024-03-24T12:14:38.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-25"}]: dispatch 2024-03-24T12:14:39.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:39 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:14:39.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi006-81775-35"}]': finished 2024-03-24T12:14:39.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-25"}]': finished 2024-03-24T12:14:39.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:39 smithi090 ceph-mon[33646]: osdmap e224: 8 total, 8 up, 8 in 2024-03-24T12:14:39.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:39 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3675363937' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:39.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:39 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2477229688' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi006-81794-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:39.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:39.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi006-81794-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:39 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:14:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi006-81775-35"}]': finished 2024-03-24T12:14:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-25"}]': finished 2024-03-24T12:14:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:39 smithi006 ceph-mon[30108]: osdmap e224: 8 total, 8 up, 8 in 2024-03-24T12:14:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3675363937' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2477229688' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi006-81794-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi006-81794-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:39.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:39 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:14:39.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi006-81775-35"}]': finished 2024-03-24T12:14:39.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-25"}]': finished 2024-03-24T12:14:39.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:39 smithi006 ceph-mon[41734]: osdmap e224: 8 total, 8 up, 8 in 2024-03-24T12:14:39.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:39 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3675363937' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:39.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:39 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2477229688' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi006-81794-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:39.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi006-81775-35"}]: dispatch 2024-03-24T12:14:39.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi006-81794-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:40.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:40 smithi090 ceph-mon[33646]: pgmap v294: 324 pgs: 32 unknown, 292 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:14:40.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:40 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi006-81775-35"}]': finished 2024-03-24T12:14:40.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:40 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi006-81794-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:40.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:40 smithi090 ceph-mon[33646]: osdmap e225: 8 total, 8 up, 8 in 2024-03-24T12:14:40.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi006-81775-36"}]: dispatch 2024-03-24T12:14:40.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi006-81775-36"}]: dispatch 2024-03-24T12:14:40.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi006-81775-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:40 smithi006 ceph-mon[30108]: pgmap v294: 324 pgs: 32 unknown, 292 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:14:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:40 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi006-81775-35"}]': finished 2024-03-24T12:14:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:40 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi006-81794-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:40 smithi006 ceph-mon[30108]: osdmap e225: 8 total, 8 up, 8 in 2024-03-24T12:14:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi006-81775-36"}]: dispatch 2024-03-24T12:14:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi006-81775-36"}]: dispatch 2024-03-24T12:14:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi006-81775-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:40.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:40 smithi006 ceph-mon[41734]: pgmap v294: 324 pgs: 32 unknown, 292 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:14:40.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:40 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi006-81775-35"}]': finished 2024-03-24T12:14:40.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:40 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi006-81794-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:40.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:40 smithi006 ceph-mon[41734]: osdmap e225: 8 total, 8 up, 8 in 2024-03-24T12:14:40.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi006-81775-36"}]: dispatch 2024-03-24T12:14:40.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi006-81775-36"}]: dispatch 2024-03-24T12:14:40.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi006-81775-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:41.612 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:41.612 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi006-81775-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:41.612 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[41734]: osdmap e226: 8 total, 8 up, 8 in 2024-03-24T12:14:41.613 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi006-81775-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi006-81775-36"}]: dispatch 2024-03-24T12:14:41.613 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:41.613 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:41.613 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1380138907' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:41.613 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:41.613 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1380138907' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:41.613 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:41.613 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1380138907' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi006-81794-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:41.613 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi006-81794-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:41.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:41.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi006-81775-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:41.614 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[30108]: osdmap e226: 8 total, 8 up, 8 in 2024-03-24T12:14:41.614 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi006-81775-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi006-81775-36"}]: dispatch 2024-03-24T12:14:41.614 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:41.614 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:41.614 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1380138907' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:41.614 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:41.614 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1380138907' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:41.614 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:41.614 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1380138907' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi006-81794-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:41.614 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:41 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi006-81794-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:41.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:41 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:41.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi006-81775-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:41.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:41 smithi090 ceph-mon[33646]: osdmap e226: 8 total, 8 up, 8 in 2024-03-24T12:14:41.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi006-81775-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi006-81775-36"}]: dispatch 2024-03-24T12:14:41.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:41.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:41 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:41.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1380138907' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:41.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:41 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:41.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1380138907' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:41.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:41 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:41.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1380138907' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi006-81794-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:41.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:41 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi006-81794-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:42.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:14:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:14:42.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:42 smithi090 ceph-mon[33646]: pgmap v297: 292 pgs: 32 unknown, 260 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:14:42.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:42 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:42.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:42 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi006-81794-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:42.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:42 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1380138907' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi006-81794-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:42.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:42 smithi090 ceph-mon[33646]: osdmap e227: 8 total, 8 up, 8 in 2024-03-24T12:14:42.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:42 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi006-81794-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:42 smithi006 ceph-mon[30108]: pgmap v297: 292 pgs: 32 unknown, 260 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:14:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:42 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:42 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi006-81794-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:42 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1380138907' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi006-81794-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:42 smithi006 ceph-mon[30108]: osdmap e227: 8 total, 8 up, 8 in 2024-03-24T12:14:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:42 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi006-81794-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:42.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:42 smithi006 ceph-mon[41734]: pgmap v297: 292 pgs: 32 unknown, 260 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:14:42.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:42 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:42.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:42 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi006-81794-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:42.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:42 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1380138907' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi006-81794-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:42.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:42 smithi006 ceph-mon[41734]: osdmap e227: 8 total, 8 up, 8 in 2024-03-24T12:14:42.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:42 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi006-81794-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:43.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:43 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:14:43.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:43 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi006-81775-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi006-81775-36"}]': finished 2024-03-24T12:14:43.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:43 smithi090 ceph-mon[33646]: osdmap e228: 8 total, 8 up, 8 in 2024-03-24T12:14:43.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:43 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:43.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:43 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:43.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:43 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:14:43.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:43 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi006-81775-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi006-81775-36"}]': finished 2024-03-24T12:14:43.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:43 smithi006 ceph-mon[30108]: osdmap e228: 8 total, 8 up, 8 in 2024-03-24T12:14:43.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:43 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:43.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:43 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:43.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:43 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:14:43.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:43 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi006-81775-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi006-81775-36"}]': finished 2024-03-24T12:14:43.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:43 smithi006 ceph-mon[41734]: osdmap e228: 8 total, 8 up, 8 in 2024-03-24T12:14:43.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:43 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:43.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:43 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:44.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:44 smithi090 ceph-mon[33646]: pgmap v300: 300 pgs: 8 unknown, 5 creating+peering, 9 creating+activating, 278 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-03-24T12:14:44.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:44 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi006-81794-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi006-81794-46"}]': finished 2024-03-24T12:14:44.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:44 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-27", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:14:44.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:44 smithi090 ceph-mon[33646]: osdmap e229: 8 total, 8 up, 8 in 2024-03-24T12:14:44.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:44 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-27"}]: dispatch 2024-03-24T12:14:44.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:44 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-27"}]: dispatch 2024-03-24T12:14:44.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:44 smithi006 ceph-mon[30108]: pgmap v300: 300 pgs: 8 unknown, 5 creating+peering, 9 creating+activating, 278 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-03-24T12:14:44.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:44 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi006-81794-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi006-81794-46"}]': finished 2024-03-24T12:14:44.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:44 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-27", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:14:44.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:44 smithi006 ceph-mon[30108]: osdmap e229: 8 total, 8 up, 8 in 2024-03-24T12:14:44.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:44 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-27"}]: dispatch 2024-03-24T12:14:44.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:44 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-27"}]: dispatch 2024-03-24T12:14:44.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:44 smithi006 ceph-mon[41734]: pgmap v300: 300 pgs: 8 unknown, 5 creating+peering, 9 creating+activating, 278 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-03-24T12:14:44.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:44 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi006-81794-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi006-81794-46"}]': finished 2024-03-24T12:14:44.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:44 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-27", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:14:44.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:44 smithi006 ceph-mon[41734]: osdmap e229: 8 total, 8 up, 8 in 2024-03-24T12:14:44.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:44 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-27"}]: dispatch 2024-03-24T12:14:44.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:44 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-27"}]: dispatch 2024-03-24T12:14:45.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:14:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:14:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:14:46.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-27"}]': finished 2024-03-24T12:14:46.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-27", "mode": "writeback"}]: dispatch 2024-03-24T12:14:46.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:45 smithi090 ceph-mon[33646]: osdmap e230: 8 total, 8 up, 8 in 2024-03-24T12:14:46.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-27", "mode": "writeback"}]: dispatch 2024-03-24T12:14:46.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi006-81775-36"}]: dispatch 2024-03-24T12:14:46.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:45 smithi090 ceph-mon[33646]: pgmap v303: 300 pgs: 8 unknown, 5 creating+peering, 9 creating+activating, 278 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:14:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-27"}]': finished 2024-03-24T12:14:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-27", "mode": "writeback"}]: dispatch 2024-03-24T12:14:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:45 smithi006 ceph-mon[30108]: osdmap e230: 8 total, 8 up, 8 in 2024-03-24T12:14:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-27", "mode": "writeback"}]: dispatch 2024-03-24T12:14:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi006-81775-36"}]: dispatch 2024-03-24T12:14:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:45 smithi006 ceph-mon[30108]: pgmap v303: 300 pgs: 8 unknown, 5 creating+peering, 9 creating+activating, 278 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:14:46.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-27"}]': finished 2024-03-24T12:14:46.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-27", "mode": "writeback"}]: dispatch 2024-03-24T12:14:46.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:45 smithi006 ceph-mon[41734]: osdmap e230: 8 total, 8 up, 8 in 2024-03-24T12:14:46.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-27", "mode": "writeback"}]: dispatch 2024-03-24T12:14:46.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi006-81775-36"}]: dispatch 2024-03-24T12:14:46.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:45 smithi006 ceph-mon[41734]: pgmap v303: 300 pgs: 8 unknown, 5 creating+peering, 9 creating+activating, 278 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:14:47.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:46 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:14:47.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:46 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-27", "mode": "writeback"}]': finished 2024-03-24T12:14:47.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:46 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi006-81775-36"}]': finished 2024-03-24T12:14:47.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:46 smithi090 ceph-mon[33646]: osdmap e231: 8 total, 8 up, 8 in 2024-03-24T12:14:47.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:46 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi006-81775-36"}]: dispatch 2024-03-24T12:14:47.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:46 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1380138907' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:47.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:46 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:47.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:46 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:14:47.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:46 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-27", "mode": "writeback"}]': finished 2024-03-24T12:14:47.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:46 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi006-81775-36"}]': finished 2024-03-24T12:14:47.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:46 smithi006 ceph-mon[30108]: osdmap e231: 8 total, 8 up, 8 in 2024-03-24T12:14:47.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:46 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi006-81775-36"}]: dispatch 2024-03-24T12:14:47.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:46 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1380138907' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:47.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:46 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:47.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:46 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:14:47.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:46 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-27", "mode": "writeback"}]': finished 2024-03-24T12:14:47.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:46 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi006-81775-36"}]': finished 2024-03-24T12:14:47.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:46 smithi006 ceph-mon[41734]: osdmap e231: 8 total, 8 up, 8 in 2024-03-24T12:14:47.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:46 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi006-81775-36"}]: dispatch 2024-03-24T12:14:47.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:46 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1380138907' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:47.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:46 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:48.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi006-81775-36"}]': finished 2024-03-24T12:14:48.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:47 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-46"}]': finished 2024-03-24T12:14:48.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:47 smithi090 ceph-mon[33646]: osdmap e232: 8 total, 8 up, 8 in 2024-03-24T12:14:48.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1380138907' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:48.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:47 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:48.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1650089288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:48.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:47 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:48.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1650089288' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:48.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:47 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:48.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1650089288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:48.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:47 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:48.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:48.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:47 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:48.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:47 smithi090 ceph-mon[33646]: pgmap v306: 292 pgs: 5 creating+activating, 287 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2024-03-24T12:14:48.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:47 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:14:48.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:47 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:14:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi006-81775-36"}]': finished 2024-03-24T12:14:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-46"}]': finished 2024-03-24T12:14:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[30108]: osdmap e232: 8 total, 8 up, 8 in 2024-03-24T12:14:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1380138907' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1650089288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1650089288' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:48.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:48.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1650089288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:48.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:48.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:48.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:48.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[30108]: pgmap v306: 292 pgs: 5 creating+activating, 287 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2024-03-24T12:14:48.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:14:48.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:14:48.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3078349878' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi006-81775-36"}]': finished 2024-03-24T12:14:48.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-46"}]': finished 2024-03-24T12:14:48.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[41734]: osdmap e232: 8 total, 8 up, 8 in 2024-03-24T12:14:48.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1380138907' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:48.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-46"}]: dispatch 2024-03-24T12:14:48.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1650089288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:48.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:48.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1650089288' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:48.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:48.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1650089288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:48.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:48.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:48.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:14:48.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[41734]: pgmap v306: 292 pgs: 5 creating+activating, 287 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2024-03-24T12:14:48.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:14:48.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:47 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:14:49.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:48 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:49.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:48 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-46"}]': finished 2024-03-24T12:14:49.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:48 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:49.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:48 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:14:49.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-27"}]: dispatch 2024-03-24T12:14:49.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1650089288' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi006-81775-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:49.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:48 smithi090 ceph-mon[33646]: osdmap e233: 8 total, 8 up, 8 in 2024-03-24T12:14:49.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:48 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-27"}]: dispatch 2024-03-24T12:14:49.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:48 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi006-81775-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:49.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-47"}]: dispatch 2024-03-24T12:14:49.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-47"}]: dispatch 2024-03-24T12:14:49.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi006-81794-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-46"}]': finished 2024-03-24T12:14:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:14:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-27"}]: dispatch 2024-03-24T12:14:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1650089288' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi006-81775-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[30108]: osdmap e233: 8 total, 8 up, 8 in 2024-03-24T12:14:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-27"}]: dispatch 2024-03-24T12:14:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi006-81775-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-47"}]: dispatch 2024-03-24T12:14:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-47"}]: dispatch 2024-03-24T12:14:49.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi006-81794-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:49.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:49.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-46"}]': finished 2024-03-24T12:14:49.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:49.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:14:49.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-27"}]: dispatch 2024-03-24T12:14:49.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1650089288' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi006-81775-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:49.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[41734]: osdmap e233: 8 total, 8 up, 8 in 2024-03-24T12:14:49.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-27"}]: dispatch 2024-03-24T12:14:49.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi006-81775-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:49.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-47"}]: dispatch 2024-03-24T12:14:49.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-47"}]: dispatch 2024-03-24T12:14:49.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi006-81794-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:50.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:49 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:14:50.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:49 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-27"}]': finished 2024-03-24T12:14:50.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:49 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi006-81794-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:50.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:49 smithi090 ceph-mon[33646]: pgmap v309: 292 pgs: 292 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-24T12:14:50.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:49 smithi090 ceph-mon[33646]: osdmap e234: 8 total, 8 up, 8 in 2024-03-24T12:14:50.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:49 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi006-81794-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi006-81794-47"}]: dispatch 2024-03-24T12:14:50.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:49 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:14:50.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:49 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-27"}]': finished 2024-03-24T12:14:50.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:49 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi006-81794-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:50.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:49 smithi006 ceph-mon[30108]: pgmap v309: 292 pgs: 292 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-24T12:14:50.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:49 smithi006 ceph-mon[30108]: osdmap e234: 8 total, 8 up, 8 in 2024-03-24T12:14:50.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:49 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi006-81794-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi006-81794-47"}]: dispatch 2024-03-24T12:14:50.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:49 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:14:50.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:49 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-27"}]': finished 2024-03-24T12:14:50.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:49 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi006-81794-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:50.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:49 smithi006 ceph-mon[41734]: pgmap v309: 292 pgs: 292 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-24T12:14:50.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:49 smithi006 ceph-mon[41734]: osdmap e234: 8 total, 8 up, 8 in 2024-03-24T12:14:50.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:49 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi006-81794-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi006-81794-47"}]: dispatch 2024-03-24T12:14:51.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:50 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi006-81775-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi006-81775-37"}]': finished 2024-03-24T12:14:51.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:50 smithi090 ceph-mon[33646]: osdmap e235: 8 total, 8 up, 8 in 2024-03-24T12:14:51.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:50 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi006-81794-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi006-81794-47"}]': finished 2024-03-24T12:14:51.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:50 smithi090 ceph-mon[33646]: osdmap e236: 8 total, 8 up, 8 in 2024-03-24T12:14:51.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:50 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:51.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:50 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:51.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:50 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi006-81775-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi006-81775-37"}]': finished 2024-03-24T12:14:51.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:50 smithi006 ceph-mon[30108]: osdmap e235: 8 total, 8 up, 8 in 2024-03-24T12:14:51.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:50 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi006-81794-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi006-81794-47"}]': finished 2024-03-24T12:14:51.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:50 smithi006 ceph-mon[30108]: osdmap e236: 8 total, 8 up, 8 in 2024-03-24T12:14:51.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:50 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:51.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:50 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:51.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:50 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi006-81775-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi006-81775-37"}]': finished 2024-03-24T12:14:51.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:50 smithi006 ceph-mon[41734]: osdmap e235: 8 total, 8 up, 8 in 2024-03-24T12:14:51.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:50 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi006-81794-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi006-81794-47"}]': finished 2024-03-24T12:14:51.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:50 smithi006 ceph-mon[41734]: osdmap e236: 8 total, 8 up, 8 in 2024-03-24T12:14:51.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:50 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:51.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:50 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:14:52.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:51 smithi090 ceph-mon[33646]: pgmap v312: 308 pgs: 48 unknown, 260 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:14:52.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:51 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:52.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:51 smithi090 ceph-mon[33646]: osdmap e237: 8 total, 8 up, 8 in 2024-03-24T12:14:52.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:51 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1650089288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:52.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:51 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:52.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:51 smithi006 ceph-mon[30108]: pgmap v312: 308 pgs: 48 unknown, 260 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:14:52.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:51 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:52.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:51 smithi006 ceph-mon[30108]: osdmap e237: 8 total, 8 up, 8 in 2024-03-24T12:14:52.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1650089288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:52.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:51 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:52.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:51 smithi006 ceph-mon[41734]: pgmap v312: 308 pgs: 48 unknown, 260 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:14:52.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:51 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:14:52.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:51 smithi006 ceph-mon[41734]: osdmap e237: 8 total, 8 up, 8 in 2024-03-24T12:14:52.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:51 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1650089288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:52.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:51 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:52.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:14:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:14:53.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:52 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:14:53.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:52 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37"}]': finished 2024-03-24T12:14:53.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:52 smithi090 ceph-mon[33646]: osdmap e238: 8 total, 8 up, 8 in 2024-03-24T12:14:53.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:52 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1650089288' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:53.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:52 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:53.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:52 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-47"}]: dispatch 2024-03-24T12:14:53.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:52 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:14:53.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:52 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37"}]': finished 2024-03-24T12:14:53.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:52 smithi006 ceph-mon[30108]: osdmap e238: 8 total, 8 up, 8 in 2024-03-24T12:14:53.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:52 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1650089288' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:53.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:52 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:53.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:52 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-47"}]: dispatch 2024-03-24T12:14:53.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:52 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:14:53.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:52 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi006-81775-37"}]': finished 2024-03-24T12:14:53.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:52 smithi006 ceph-mon[41734]: osdmap e238: 8 total, 8 up, 8 in 2024-03-24T12:14:53.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:52 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1650089288' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:53.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:52 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi006-81775-37"}]: dispatch 2024-03-24T12:14:53.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:52 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-47"}]: dispatch 2024-03-24T12:14:54.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:53 smithi006 ceph-mon[30108]: pgmap v315: 292 pgs: 292 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:14:54.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:53 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi006-81775-37"}]': finished 2024-03-24T12:14:54.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:53 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-47"}]': finished 2024-03-24T12:14:54.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:53 smithi006 ceph-mon[30108]: osdmap e239: 8 total, 8 up, 8 in 2024-03-24T12:14:54.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:53 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-47"}]: dispatch 2024-03-24T12:14:54.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:53 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:54.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:53 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:54.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:53 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi006-81775-38"}]: dispatch 2024-03-24T12:14:54.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:53 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi006-81775-38"}]: dispatch 2024-03-24T12:14:54.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:53 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi006-81775-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:54.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:53 smithi006 ceph-mon[41734]: pgmap v315: 292 pgs: 292 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:14:54.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:53 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi006-81775-37"}]': finished 2024-03-24T12:14:54.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:53 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-47"}]': finished 2024-03-24T12:14:54.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:53 smithi006 ceph-mon[41734]: osdmap e239: 8 total, 8 up, 8 in 2024-03-24T12:14:54.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:53 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-47"}]: dispatch 2024-03-24T12:14:54.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:53 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:54.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:53 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:54.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:53 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi006-81775-38"}]: dispatch 2024-03-24T12:14:54.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:53 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi006-81775-38"}]: dispatch 2024-03-24T12:14:54.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:53 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi006-81775-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:54.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:53 smithi090 ceph-mon[33646]: pgmap v315: 292 pgs: 292 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:14:54.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:53 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi006-81775-37"}]': finished 2024-03-24T12:14:54.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:53 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi006-81794-47"}]': finished 2024-03-24T12:14:54.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:53 smithi090 ceph-mon[33646]: osdmap e239: 8 total, 8 up, 8 in 2024-03-24T12:14:54.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:53 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-47"}]: dispatch 2024-03-24T12:14:54.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:53 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:54.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:53 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:14:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:53 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi006-81775-38"}]: dispatch 2024-03-24T12:14:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:53 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi006-81775-38"}]: dispatch 2024-03-24T12:14:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:53 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi006-81775-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:55.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:55.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:14:55.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:14:55.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:14:55.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-47"}]': finished 2024-03-24T12:14:55.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-29", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:14:55.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi006-81775-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:55.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[30108]: osdmap e240: 8 total, 8 up, 8 in 2024-03-24T12:14:55.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-29"}]: dispatch 2024-03-24T12:14:55.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi006-81775-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi006-81775-38"}]: dispatch 2024-03-24T12:14:55.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-29"}]: dispatch 2024-03-24T12:14:55.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/749525052' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:14:55.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:14:55.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/749525052' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:14:55.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:14:55.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/749525052' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi006-81794-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:55.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi006-81794-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:55.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:55.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:14:55.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:14:55.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:14:55.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-47"}]': finished 2024-03-24T12:14:55.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-29", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:14:55.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi006-81775-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:55.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[41734]: osdmap e240: 8 total, 8 up, 8 in 2024-03-24T12:14:55.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-29"}]: dispatch 2024-03-24T12:14:55.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi006-81775-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi006-81775-38"}]: dispatch 2024-03-24T12:14:55.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-29"}]: dispatch 2024-03-24T12:14:55.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/749525052' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:14:55.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:14:55.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/749525052' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:14:55.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:14:55.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/749525052' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi006-81794-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:55.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:54 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi006-81794-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:55.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:54 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:14:55.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:54 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:14:55.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:54 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:14:55.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:54 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:14:55.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/270212032' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi006-81794-47"}]': finished 2024-03-24T12:14:55.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:54 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-29", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:14:55.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi006-81775-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:55.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:54 smithi090 ceph-mon[33646]: osdmap e240: 8 total, 8 up, 8 in 2024-03-24T12:14:55.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-29"}]: dispatch 2024-03-24T12:14:55.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi006-81775-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi006-81775-38"}]: dispatch 2024-03-24T12:14:55.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:54 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-29"}]: dispatch 2024-03-24T12:14:55.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/749525052' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:14:55.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:54 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:14:55.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/749525052' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:14:55.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:54 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:14:55.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/749525052' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi006-81794-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:55.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:54 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi006-81794-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:14:55.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:14:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:14:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:14:56.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:56 smithi090 ceph-mon[33646]: pgmap v318: 292 pgs: 292 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-24T12:14:56.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:56 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:14:56.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:56 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:14:56.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:56 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:14:56.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:56 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-29"}]': finished 2024-03-24T12:14:56.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:56 smithi006 ceph-mon[30108]: pgmap v318: 292 pgs: 292 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-24T12:14:56.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:56 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:14:56.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:56 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:14:56.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:56 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:14:56.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:56 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-29"}]': finished 2024-03-24T12:14:56.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:56 smithi006 ceph-mon[41734]: pgmap v318: 292 pgs: 292 active+clean; 4.6 MiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-24T12:14:56.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:56 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:14:56.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:56 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:14:56.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:56 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:14:56.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:56 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-29"}]': finished 2024-03-24T12:14:57.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:57 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi006-81794-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:57.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:57 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-29", "mode": "writeback"}]: dispatch 2024-03-24T12:14:57.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:57 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/749525052' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi006-81794-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:14:57.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:57 smithi090 ceph-mon[33646]: osdmap e241: 8 total, 8 up, 8 in 2024-03-24T12:14:57.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:57 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-29", "mode": "writeback"}]: dispatch 2024-03-24T12:14:57.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:57 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi006-81794-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:14:57.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:57 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:14:57.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:57 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi006-81775-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi006-81775-38"}]': finished 2024-03-24T12:14:57.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:57 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi006-81794-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:57.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:57 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-29", "mode": "writeback"}]: dispatch 2024-03-24T12:14:57.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:57 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/749525052' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi006-81794-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:14:57.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:57 smithi006 ceph-mon[30108]: osdmap e241: 8 total, 8 up, 8 in 2024-03-24T12:14:57.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:57 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-29", "mode": "writeback"}]: dispatch 2024-03-24T12:14:57.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:57 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi006-81794-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:14:57.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:57 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:14:57.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:57 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi006-81775-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi006-81775-38"}]': finished 2024-03-24T12:14:57.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:57 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi006-81794-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:14:57.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:57 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-29", "mode": "writeback"}]: dispatch 2024-03-24T12:14:57.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:57 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/749525052' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi006-81794-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:14:57.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:57 smithi006 ceph-mon[41734]: osdmap e241: 8 total, 8 up, 8 in 2024-03-24T12:14:57.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:57 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-29", "mode": "writeback"}]: dispatch 2024-03-24T12:14:57.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:57 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi006-81794-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:14:57.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:57 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:14:57.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:57 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi006-81775-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi006-81775-38"}]': finished 2024-03-24T12:14:58.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:58 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-29", "mode": "writeback"}]': finished 2024-03-24T12:14:58.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:58 smithi090 ceph-mon[33646]: pgmap v321: 300 pgs: 2 creating+peering, 6 unknown, 292 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:14:58.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:58 smithi090 ceph-mon[33646]: osdmap e242: 8 total, 8 up, 8 in 2024-03-24T12:14:58.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:58 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi006-81794-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi006-81794-48"}]': finished 2024-03-24T12:14:58.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:58 smithi090 ceph-mon[33646]: osdmap e243: 8 total, 8 up, 8 in 2024-03-24T12:14:58.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:58 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-29", "mode": "writeback"}]': finished 2024-03-24T12:14:58.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:58 smithi006 ceph-mon[30108]: pgmap v321: 300 pgs: 2 creating+peering, 6 unknown, 292 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:14:58.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:58 smithi006 ceph-mon[30108]: osdmap e242: 8 total, 8 up, 8 in 2024-03-24T12:14:58.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:58 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi006-81794-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi006-81794-48"}]': finished 2024-03-24T12:14:58.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:58 smithi006 ceph-mon[30108]: osdmap e243: 8 total, 8 up, 8 in 2024-03-24T12:14:58.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:58 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-29", "mode": "writeback"}]': finished 2024-03-24T12:14:58.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:58 smithi006 ceph-mon[41734]: pgmap v321: 300 pgs: 2 creating+peering, 6 unknown, 292 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:14:58.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:58 smithi006 ceph-mon[41734]: osdmap e242: 8 total, 8 up, 8 in 2024-03-24T12:14:58.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:58 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi006-81794-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi006-81794-48"}]': finished 2024-03-24T12:14:58.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:58 smithi006 ceph-mon[41734]: osdmap e243: 8 total, 8 up, 8 in 2024-03-24T12:15:00.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:59 smithi090 ceph-mon[33646]: osdmap e244: 8 total, 8 up, 8 in 2024-03-24T12:15:00.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:59 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi006-81775-38"}]: dispatch 2024-03-24T12:15:00.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:59 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:00.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:59 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:00.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:14:59 smithi090 ceph-mon[33646]: pgmap v324: 300 pgs: 8 unknown, 292 active+clean; 8.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-24T12:15:00.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:59 smithi006 ceph-mon[41734]: osdmap e244: 8 total, 8 up, 8 in 2024-03-24T12:15:00.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:59 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi006-81775-38"}]: dispatch 2024-03-24T12:15:00.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:59 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:00.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:59 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:00.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:14:59 smithi006 ceph-mon[41734]: pgmap v324: 300 pgs: 8 unknown, 292 active+clean; 8.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-24T12:15:00.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:59 smithi006 ceph-mon[30108]: osdmap e244: 8 total, 8 up, 8 in 2024-03-24T12:15:00.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:59 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi006-81775-38"}]: dispatch 2024-03-24T12:15:00.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:59 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:00.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:59 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:00.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:14:59 smithi006 ceph-mon[30108]: pgmap v324: 300 pgs: 8 unknown, 292 active+clean; 8.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-24T12:15:01.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:00 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi006-81775-38"}]': finished 2024-03-24T12:15:01.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:00 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:15:01.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:00 smithi090 ceph-mon[33646]: osdmap e245: 8 total, 8 up, 8 in 2024-03-24T12:15:01.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:00 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-29"}]: dispatch 2024-03-24T12:15:01.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:00 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi006-81775-38"}]: dispatch 2024-03-24T12:15:01.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:00 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/749525052' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:15:01.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:00 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-29"}]: dispatch 2024-03-24T12:15:01.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:00 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:15:01.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:00 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:01.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:00 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi006-81775-38"}]': finished 2024-03-24T12:15:01.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:00 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:15:01.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:00 smithi006 ceph-mon[30108]: osdmap e245: 8 total, 8 up, 8 in 2024-03-24T12:15:01.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:00 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-29"}]: dispatch 2024-03-24T12:15:01.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:00 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi006-81775-38"}]: dispatch 2024-03-24T12:15:01.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:00 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/749525052' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:15:01.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:00 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-29"}]: dispatch 2024-03-24T12:15:01.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:00 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:15:01.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:00 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:01.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:00 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi006-81775-38"}]': finished 2024-03-24T12:15:01.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:00 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:15:01.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:00 smithi006 ceph-mon[41734]: osdmap e245: 8 total, 8 up, 8 in 2024-03-24T12:15:01.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:00 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-29"}]: dispatch 2024-03-24T12:15:01.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:00 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi006-81775-38"}]: dispatch 2024-03-24T12:15:01.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:00 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/749525052' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:15:01.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:00 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-29"}]: dispatch 2024-03-24T12:15:01.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:00 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:15:01.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:00 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:02.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:15:02.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi006-81775-38"}]': finished 2024-03-24T12:15:02.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-29"}]': finished 2024-03-24T12:15:02.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi006-81794-48"}]': finished 2024-03-24T12:15:02.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: osdmap e246: 8 total, 8 up, 8 in 2024-03-24T12:15:02.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/749525052' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:15:02.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:15:02.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/377467371' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:02.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:02.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/377467371' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:02.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:02.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/377467371' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi006-81775-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:02.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi006-81775-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:02.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: pgmap v327: 292 pgs: 292 active+clean; 8.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-24T12:15:02.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.7", "id": [3, 7]}]: dispatch 2024-03-24T12:15:02.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.8", "id": [3, 7, 6, 1]}]: dispatch 2024-03-24T12:15:02.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.7", "id": [3, 7]}]: dispatch 2024-03-24T12:15:02.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.a", "id": [3, 1]}]: dispatch 2024-03-24T12:15:02.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.8", "id": [3, 7, 6, 1]}]: dispatch 2024-03-24T12:15:02.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.b", "id": [5, 1]}]: dispatch 2024-03-24T12:15:02.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.a", "id": [3, 1]}]: dispatch 2024-03-24T12:15:02.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.b", "id": [5, 1]}]: dispatch 2024-03-24T12:15:02.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi006-81794-48"}]': finished 2024-03-24T12:15:02.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi006-81775-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:02.180 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.7", "id": [3, 7]}]': finished 2024-03-24T12:15:02.181 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.8", "id": [3, 7, 6, 1]}]': finished 2024-03-24T12:15:02.181 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.a", "id": [3, 1]}]': finished 2024-03-24T12:15:02.181 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.b", "id": [5, 1]}]': finished 2024-03-24T12:15:02.181 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:01 smithi090 ceph-mon[33646]: osdmap e247: 8 total, 8 up, 8 in 2024-03-24T12:15:02.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:15:02.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi006-81775-38"}]': finished 2024-03-24T12:15:02.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-29"}]': finished 2024-03-24T12:15:02.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi006-81794-48"}]': finished 2024-03-24T12:15:02.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: osdmap e246: 8 total, 8 up, 8 in 2024-03-24T12:15:02.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/749525052' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:15:02.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:15:02.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/377467371' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:02.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:02.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/377467371' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:02.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:02.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/377467371' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi006-81775-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:02.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi006-81775-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:02.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: pgmap v327: 292 pgs: 292 active+clean; 8.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-24T12:15:02.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.7", "id": [3, 7]}]: dispatch 2024-03-24T12:15:02.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.8", "id": [3, 7, 6, 1]}]: dispatch 2024-03-24T12:15:02.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.7", "id": [3, 7]}]: dispatch 2024-03-24T12:15:02.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.a", "id": [3, 1]}]: dispatch 2024-03-24T12:15:02.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.8", "id": [3, 7, 6, 1]}]: dispatch 2024-03-24T12:15:02.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.b", "id": [5, 1]}]: dispatch 2024-03-24T12:15:02.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.a", "id": [3, 1]}]: dispatch 2024-03-24T12:15:02.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:15:02.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2035463557' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi006-81775-38"}]': finished 2024-03-24T12:15:02.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-29"}]': finished 2024-03-24T12:15:02.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi006-81794-48"}]': finished 2024-03-24T12:15:02.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: osdmap e246: 8 total, 8 up, 8 in 2024-03-24T12:15:02.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/749525052' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:15:02.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi006-81794-48"}]: dispatch 2024-03-24T12:15:02.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/377467371' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:02.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:02.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/377467371' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:02.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:02.215 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/377467371' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi006-81775-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:02.215 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi006-81775-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:02.215 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: pgmap v327: 292 pgs: 292 active+clean; 8.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-24T12:15:02.215 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.7", "id": [3, 7]}]: dispatch 2024-03-24T12:15:02.215 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.8", "id": [3, 7, 6, 1]}]: dispatch 2024-03-24T12:15:02.215 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.7", "id": [3, 7]}]: dispatch 2024-03-24T12:15:02.215 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.a", "id": [3, 1]}]: dispatch 2024-03-24T12:15:02.215 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.8", "id": [3, 7, 6, 1]}]: dispatch 2024-03-24T12:15:02.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.b", "id": [5, 1]}]: dispatch 2024-03-24T12:15:02.216 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.a", "id": [3, 1]}]: dispatch 2024-03-24T12:15:02.216 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.b", "id": [5, 1]}]: dispatch 2024-03-24T12:15:02.216 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi006-81794-48"}]': finished 2024-03-24T12:15:02.216 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi006-81775-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:02.216 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.7", "id": [3, 7]}]': finished 2024-03-24T12:15:02.216 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.8", "id": [3, 7, 6, 1]}]': finished 2024-03-24T12:15:02.216 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.a", "id": [3, 1]}]': finished 2024-03-24T12:15:02.217 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.b", "id": [5, 1]}]': finished 2024-03-24T12:15:02.217 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[30108]: osdmap e247: 8 total, 8 up, 8 in 2024-03-24T12:15:02.217 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.b", "id": [5, 1]}]: dispatch 2024-03-24T12:15:02.217 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi006-81794-48"}]': finished 2024-03-24T12:15:02.217 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi006-81775-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:02.217 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.7", "id": [3, 7]}]': finished 2024-03-24T12:15:02.217 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.8", "id": [3, 7, 6, 1]}]': finished 2024-03-24T12:15:02.217 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.a", "id": [3, 1]}]': finished 2024-03-24T12:15:02.217 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "39.b", "id": [5, 1]}]': finished 2024-03-24T12:15:02.217 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:01 smithi006 ceph-mon[41734]: osdmap e247: 8 total, 8 up, 8 in 2024-03-24T12:15:02.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:15:02 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:15:03.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/377467371' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi006-81775-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:03.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:03 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi006-81775-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:03.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi006-81794-49"}]: dispatch 2024-03-24T12:15:03.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi006-81794-49"}]: dispatch 2024-03-24T12:15:03.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi006-81794-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:03.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:03 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:15:03.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:03 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:15:03.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:03 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:15:03.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi006-81794-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:03.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/377467371' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi006-81775-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:03.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:03 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi006-81775-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:03.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi006-81794-49"}]: dispatch 2024-03-24T12:15:03.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi006-81794-49"}]: dispatch 2024-03-24T12:15:03.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi006-81794-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:03.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:03 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:15:03.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:03 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:15:03.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:03 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:15:03.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi006-81794-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:03.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/377467371' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi006-81775-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:03.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:03 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi006-81775-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:03.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi006-81794-49"}]: dispatch 2024-03-24T12:15:03.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi006-81794-49"}]: dispatch 2024-03-24T12:15:03.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi006-81794-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:03.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:03 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:15:03.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:03 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:15:03.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:03 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:15:03.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi006-81794-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:04.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:04 smithi090 ceph-mon[33646]: pgmap v330: 292 pgs: 32 unknown, 4 peering, 256 active+clean; 4.6 MiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:15:04.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:04 smithi090 ceph-mon[33646]: osdmap e248: 8 total, 8 up, 8 in 2024-03-24T12:15:04.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:04 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi006-81794-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi006-81794-49"}]: dispatch 2024-03-24T12:15:04.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:04 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:04.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:04 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:04.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:04 smithi090 ceph-mon[33646]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-03-24T12:15:04.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:04 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi006-81775-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi006-81775-39"}]': finished 2024-03-24T12:15:04.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:04 smithi006 ceph-mon[30108]: pgmap v330: 292 pgs: 32 unknown, 4 peering, 256 active+clean; 4.6 MiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:15:04.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:04 smithi006 ceph-mon[30108]: osdmap e248: 8 total, 8 up, 8 in 2024-03-24T12:15:04.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:04 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi006-81794-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi006-81794-49"}]: dispatch 2024-03-24T12:15:04.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:04 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:04.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:04 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:04.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:04 smithi006 ceph-mon[30108]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-03-24T12:15:04.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:04 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi006-81775-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi006-81775-39"}]': finished 2024-03-24T12:15:04.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:04 smithi006 ceph-mon[41734]: pgmap v330: 292 pgs: 32 unknown, 4 peering, 256 active+clean; 4.6 MiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:15:04.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:04 smithi006 ceph-mon[41734]: osdmap e248: 8 total, 8 up, 8 in 2024-03-24T12:15:04.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:04 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi006-81794-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi006-81794-49"}]: dispatch 2024-03-24T12:15:04.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:04 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:04.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:04 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:04.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:04 smithi006 ceph-mon[41734]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-03-24T12:15:04.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:04 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi006-81775-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi006-81775-39"}]': finished 2024-03-24T12:15:05.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:05 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:05.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:05 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:05.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:05 smithi090 ceph-mon[33646]: osdmap e249: 8 total, 8 up, 8 in 2024-03-24T12:15:05.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:05 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:05.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:05 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi006-81794-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi006-81794-49"}]': finished 2024-03-24T12:15:05.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:05 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-31", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:15:05.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:05 smithi090 ceph-mon[33646]: osdmap e250: 8 total, 8 up, 8 in 2024-03-24T12:15:05.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:05 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-31"}]: dispatch 2024-03-24T12:15:05.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:05 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-31"}]: dispatch 2024-03-24T12:15:05.461 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:15:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:15:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:15:05.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:05 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:05.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:05 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:05.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:05 smithi006 ceph-mon[30108]: osdmap e249: 8 total, 8 up, 8 in 2024-03-24T12:15:05.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:05 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:05.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:05 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi006-81794-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi006-81794-49"}]': finished 2024-03-24T12:15:05.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:05 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-31", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:15:05.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:05 smithi006 ceph-mon[30108]: osdmap e250: 8 total, 8 up, 8 in 2024-03-24T12:15:05.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:05 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-31"}]: dispatch 2024-03-24T12:15:05.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:05 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-31"}]: dispatch 2024-03-24T12:15:05.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:05 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:05.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:05 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:05.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:05 smithi006 ceph-mon[41734]: osdmap e249: 8 total, 8 up, 8 in 2024-03-24T12:15:05.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:05 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:05.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:05 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi006-81794-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi006-81794-49"}]': finished 2024-03-24T12:15:05.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:05 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-31", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:15:05.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:05 smithi006 ceph-mon[41734]: osdmap e250: 8 total, 8 up, 8 in 2024-03-24T12:15:05.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:05 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-31"}]: dispatch 2024-03-24T12:15:05.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:05 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-31"}]: dispatch 2024-03-24T12:15:06.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:06 smithi090 ceph-mon[33646]: pgmap v333: 308 pgs: 48 unknown, 4 peering, 256 active+clean; 4.6 MiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:15:06.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:06 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-31"}]': finished 2024-03-24T12:15:06.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:06 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-31", "mode": "writeback"}]: dispatch 2024-03-24T12:15:06.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:06 smithi090 ceph-mon[33646]: osdmap e251: 8 total, 8 up, 8 in 2024-03-24T12:15:06.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:06 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-31", "mode": "writeback"}]: dispatch 2024-03-24T12:15:06.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:06 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/377467371' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:06.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:06 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:06.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:06 smithi090 ceph-mon[33646]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:06.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:06 smithi006 ceph-mon[30108]: pgmap v333: 308 pgs: 48 unknown, 4 peering, 256 active+clean; 4.6 MiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:15:06.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:06 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-31"}]': finished 2024-03-24T12:15:06.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:06 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-31", "mode": "writeback"}]: dispatch 2024-03-24T12:15:06.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:06 smithi006 ceph-mon[30108]: osdmap e251: 8 total, 8 up, 8 in 2024-03-24T12:15:06.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:06 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-31", "mode": "writeback"}]: dispatch 2024-03-24T12:15:06.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:06 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/377467371' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:06.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:06 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:06.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:06 smithi006 ceph-mon[30108]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:06.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:06 smithi006 ceph-mon[41734]: pgmap v333: 308 pgs: 48 unknown, 4 peering, 256 active+clean; 4.6 MiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:15:06.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:06 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-31"}]': finished 2024-03-24T12:15:06.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:06 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-31", "mode": "writeback"}]: dispatch 2024-03-24T12:15:06.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:06 smithi006 ceph-mon[41734]: osdmap e251: 8 total, 8 up, 8 in 2024-03-24T12:15:06.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:06 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-31", "mode": "writeback"}]: dispatch 2024-03-24T12:15:06.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:06 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/377467371' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:06.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:06 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:06.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:06 smithi006 ceph-mon[41734]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:07.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:07 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:15:07.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:07 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-31", "mode": "writeback"}]': finished 2024-03-24T12:15:07.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:07 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi006-81775-39"}]': finished 2024-03-24T12:15:07.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:07 smithi090 ceph-mon[33646]: osdmap e252: 8 total, 8 up, 8 in 2024-03-24T12:15:07.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:07 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/377467371' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:07.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:07 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi006-81794-49"}]: dispatch 2024-03-24T12:15:07.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:07 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:07.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:07 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:07.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:07 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:07.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:07 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:15:07.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:07 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-31", "mode": "writeback"}]': finished 2024-03-24T12:15:07.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:07 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi006-81775-39"}]': finished 2024-03-24T12:15:07.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:07 smithi006 ceph-mon[30108]: osdmap e252: 8 total, 8 up, 8 in 2024-03-24T12:15:07.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:07 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/377467371' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:07.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:07 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi006-81794-49"}]: dispatch 2024-03-24T12:15:07.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:07 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:07.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:07 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:07.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:07 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:07.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:07 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:15:07.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:07 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-31", "mode": "writeback"}]': finished 2024-03-24T12:15:07.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:07 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi006-81775-39"}]': finished 2024-03-24T12:15:07.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:07 smithi006 ceph-mon[41734]: osdmap e252: 8 total, 8 up, 8 in 2024-03-24T12:15:07.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:07 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/377467371' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:07.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:07 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi006-81794-49"}]: dispatch 2024-03-24T12:15:07.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:07 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi006-81775-39"}]: dispatch 2024-03-24T12:15:07.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:07 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:07.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:07 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:08.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:08 smithi090 ceph-mon[33646]: pgmap v336: 292 pgs: 19 unknown, 273 active+clean; 4.6 MiB data, 580 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-24T12:15:08.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:08 smithi090 ceph-mon[33646]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-03-24T12:15:08.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi006-81794-49"}]': finished 2024-03-24T12:15:08.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi006-81775-39"}]': finished 2024-03-24T12:15:08.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:15:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-31"}]: dispatch 2024-03-24T12:15:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:08 smithi090 ceph-mon[33646]: osdmap e253: 8 total, 8 up, 8 in 2024-03-24T12:15:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi006-81794-49"}]: dispatch 2024-03-24T12:15:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-31"}]: dispatch 2024-03-24T12:15:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi006-81775-40"}]: dispatch 2024-03-24T12:15:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi006-81775-40"}]: dispatch 2024-03-24T12:15:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi006-81775-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[30108]: pgmap v336: 292 pgs: 19 unknown, 273 active+clean; 4.6 MiB data, 580 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-24T12:15:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[30108]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-03-24T12:15:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi006-81794-49"}]': finished 2024-03-24T12:15:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi006-81775-39"}]': finished 2024-03-24T12:15:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:15:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-31"}]: dispatch 2024-03-24T12:15:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[30108]: osdmap e253: 8 total, 8 up, 8 in 2024-03-24T12:15:08.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi006-81794-49"}]: dispatch 2024-03-24T12:15:08.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-31"}]: dispatch 2024-03-24T12:15:08.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi006-81775-40"}]: dispatch 2024-03-24T12:15:08.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi006-81775-40"}]: dispatch 2024-03-24T12:15:08.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi006-81775-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:08.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[41734]: pgmap v336: 292 pgs: 19 unknown, 273 active+clean; 4.6 MiB data, 580 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-24T12:15:08.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[41734]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-03-24T12:15:08.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi006-81794-49"}]': finished 2024-03-24T12:15:08.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi006-81775-39"}]': finished 2024-03-24T12:15:08.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:15:08.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-31"}]: dispatch 2024-03-24T12:15:08.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[41734]: osdmap e253: 8 total, 8 up, 8 in 2024-03-24T12:15:08.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi006-81794-49"}]: dispatch 2024-03-24T12:15:08.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-31"}]: dispatch 2024-03-24T12:15:08.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi006-81775-40"}]: dispatch 2024-03-24T12:15:08.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi006-81775-40"}]: dispatch 2024-03-24T12:15:08.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi006-81775-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:09.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:09 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:15:09.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:09 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi006-81794-49"}]': finished 2024-03-24T12:15:09.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:09 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-31"}]': finished 2024-03-24T12:15:09.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:09 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi006-81775-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:09.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:09 smithi090 ceph-mon[33646]: osdmap e254: 8 total, 8 up, 8 in 2024-03-24T12:15:09.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:09 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi006-81775-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi006-81775-40"}]: dispatch 2024-03-24T12:15:09.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:09 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi006-81794-50"}]: dispatch 2024-03-24T12:15:09.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:09 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi006-81794-50"}]: dispatch 2024-03-24T12:15:09.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:09 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi006-81794-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:09 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:15:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:09 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi006-81794-49"}]': finished 2024-03-24T12:15:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:09 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-31"}]': finished 2024-03-24T12:15:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:09 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi006-81775-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:09 smithi006 ceph-mon[30108]: osdmap e254: 8 total, 8 up, 8 in 2024-03-24T12:15:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:09 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi006-81775-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi006-81775-40"}]: dispatch 2024-03-24T12:15:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:09 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi006-81794-50"}]: dispatch 2024-03-24T12:15:09.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:09 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi006-81794-50"}]: dispatch 2024-03-24T12:15:09.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:09 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi006-81794-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:09.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:09 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:15:09.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:09 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2650106094' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi006-81794-49"}]': finished 2024-03-24T12:15:09.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:09 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-31"}]': finished 2024-03-24T12:15:09.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:09 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi006-81775-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:09.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:09 smithi006 ceph-mon[41734]: osdmap e254: 8 total, 8 up, 8 in 2024-03-24T12:15:09.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:09 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi006-81775-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi006-81775-40"}]: dispatch 2024-03-24T12:15:09.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:09 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi006-81794-50"}]: dispatch 2024-03-24T12:15:09.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:09 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi006-81794-50"}]: dispatch 2024-03-24T12:15:09.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:09 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi006-81794-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:10.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:10 smithi090 ceph-mon[33646]: pgmap v339: 292 pgs: 292 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:15:10.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi006-81794-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:10.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:10 smithi090 ceph-mon[33646]: osdmap e255: 8 total, 8 up, 8 in 2024-03-24T12:15:10.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi006-81794-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi006-81794-50"}]: dispatch 2024-03-24T12:15:10.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:10 smithi006 ceph-mon[30108]: pgmap v339: 292 pgs: 292 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:15:10.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi006-81794-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:10.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:10 smithi006 ceph-mon[30108]: osdmap e255: 8 total, 8 up, 8 in 2024-03-24T12:15:10.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi006-81794-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi006-81794-50"}]: dispatch 2024-03-24T12:15:10.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:10 smithi006 ceph-mon[41734]: pgmap v339: 292 pgs: 292 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:15:10.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi006-81794-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:10.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:10 smithi006 ceph-mon[41734]: osdmap e255: 8 total, 8 up, 8 in 2024-03-24T12:15:10.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi006-81794-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi006-81794-50"}]: dispatch 2024-03-24T12:15:12.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi006-81775-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi006-81775-40"}]': finished 2024-03-24T12:15:12.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:11 smithi090 ceph-mon[33646]: pgmap v342: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:12.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:11 smithi090 ceph-mon[33646]: osdmap e256: 8 total, 8 up, 8 in 2024-03-24T12:15:12.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:12.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:12.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi006-81775-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi006-81775-40"}]': finished 2024-03-24T12:15:12.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:11 smithi006 ceph-mon[30108]: pgmap v342: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:12.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:11 smithi006 ceph-mon[30108]: osdmap e256: 8 total, 8 up, 8 in 2024-03-24T12:15:12.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:12.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:12.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi006-81775-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi006-81775-40"}]': finished 2024-03-24T12:15:12.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:11 smithi006 ceph-mon[41734]: pgmap v342: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:12.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:11 smithi006 ceph-mon[41734]: osdmap e256: 8 total, 8 up, 8 in 2024-03-24T12:15:12.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:12.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:12.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:15:12 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:15:13.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:12 smithi090 ceph-mon[33646]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:13.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:12 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi006-81794-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi006-81794-50"}]': finished 2024-03-24T12:15:13.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:13.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:12 smithi090 ceph-mon[33646]: osdmap e257: 8 total, 8 up, 8 in 2024-03-24T12:15:13.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:12 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:13.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:13.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:12 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:15:13.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:12 smithi006 ceph-mon[30108]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:13.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:12 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi006-81794-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi006-81794-50"}]': finished 2024-03-24T12:15:13.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:13.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:12 smithi006 ceph-mon[30108]: osdmap e257: 8 total, 8 up, 8 in 2024-03-24T12:15:13.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:12 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:13.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:13.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:12 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:15:13.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:12 smithi006 ceph-mon[41734]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:13.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:12 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi006-81794-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi006-81794-50"}]': finished 2024-03-24T12:15:13.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:13.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:12 smithi006 ceph-mon[41734]: osdmap e257: 8 total, 8 up, 8 in 2024-03-24T12:15:13.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:12 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:13.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:13.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:12 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:15:14.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:13 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-33", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:15:14.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:13 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-33"}]: dispatch 2024-03-24T12:15:14.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:13 smithi090 ceph-mon[33646]: osdmap e258: 8 total, 8 up, 8 in 2024-03-24T12:15:14.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:13 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-33"}]: dispatch 2024-03-24T12:15:14.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:13 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi006-81775-40"}]: dispatch 2024-03-24T12:15:14.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:13 smithi090 ceph-mon[33646]: pgmap v345: 300 pgs: 8 creating+peering, 292 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-24T12:15:14.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:13 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-33", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:15:14.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:13 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-33"}]: dispatch 2024-03-24T12:15:14.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:13 smithi006 ceph-mon[30108]: osdmap e258: 8 total, 8 up, 8 in 2024-03-24T12:15:14.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:13 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-33"}]: dispatch 2024-03-24T12:15:14.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:13 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi006-81775-40"}]: dispatch 2024-03-24T12:15:14.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:13 smithi006 ceph-mon[30108]: pgmap v345: 300 pgs: 8 creating+peering, 292 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-24T12:15:14.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:13 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-33", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:15:14.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:13 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-33"}]: dispatch 2024-03-24T12:15:14.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:13 smithi006 ceph-mon[41734]: osdmap e258: 8 total, 8 up, 8 in 2024-03-24T12:15:14.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:13 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-33"}]: dispatch 2024-03-24T12:15:14.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:13 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi006-81775-40"}]: dispatch 2024-03-24T12:15:14.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:13 smithi006 ceph-mon[41734]: pgmap v345: 300 pgs: 8 creating+peering, 292 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-24T12:15:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:14 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-33"}]': finished 2024-03-24T12:15:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi006-81775-40"}]': finished 2024-03-24T12:15:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:14 smithi090 ceph-mon[33646]: osdmap e259: 8 total, 8 up, 8 in 2024-03-24T12:15:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-33", "mode": "writeback"}]: dispatch 2024-03-24T12:15:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi006-81775-40"}]: dispatch 2024-03-24T12:15:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:14 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-33", "mode": "writeback"}]: dispatch 2024-03-24T12:15:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi006-81794-50"}]: dispatch 2024-03-24T12:15:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:14 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-33"}]': finished 2024-03-24T12:15:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi006-81775-40"}]': finished 2024-03-24T12:15:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:14 smithi006 ceph-mon[30108]: osdmap e259: 8 total, 8 up, 8 in 2024-03-24T12:15:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-33", "mode": "writeback"}]: dispatch 2024-03-24T12:15:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi006-81775-40"}]: dispatch 2024-03-24T12:15:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:14 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-33", "mode": "writeback"}]: dispatch 2024-03-24T12:15:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi006-81794-50"}]: dispatch 2024-03-24T12:15:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:14 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-33"}]': finished 2024-03-24T12:15:15.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi006-81775-40"}]': finished 2024-03-24T12:15:15.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:14 smithi006 ceph-mon[41734]: osdmap e259: 8 total, 8 up, 8 in 2024-03-24T12:15:15.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-33", "mode": "writeback"}]: dispatch 2024-03-24T12:15:15.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi006-81775-40"}]: dispatch 2024-03-24T12:15:15.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:14 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-33", "mode": "writeback"}]: dispatch 2024-03-24T12:15:15.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi006-81794-50"}]: dispatch 2024-03-24T12:15:15.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:15:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:15:16.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:15 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:15:16.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi006-81775-40"}]': finished 2024-03-24T12:15:16.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:15 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-33", "mode": "writeback"}]': finished 2024-03-24T12:15:16.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi006-81794-50"}]': finished 2024-03-24T12:15:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:15 smithi090 ceph-mon[33646]: osdmap e260: 8 total, 8 up, 8 in 2024-03-24T12:15:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi006-81794-50"}]: dispatch 2024-03-24T12:15:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1291879078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:15 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1291879078' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:15 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1291879078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi006-81775-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:15 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi006-81775-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:15 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:16.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:15 smithi090 ceph-mon[33646]: pgmap v348: 292 pgs: 292 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:16.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:15:16.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi006-81775-40"}]': finished 2024-03-24T12:15:16.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-33", "mode": "writeback"}]': finished 2024-03-24T12:15:16.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi006-81794-50"}]': finished 2024-03-24T12:15:16.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[30108]: osdmap e260: 8 total, 8 up, 8 in 2024-03-24T12:15:16.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi006-81794-50"}]: dispatch 2024-03-24T12:15:16.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1291879078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:16.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:16.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1291879078' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:16.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:16.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1291879078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi006-81775-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:16.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi006-81775-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:16.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:16.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:16.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[30108]: pgmap v348: 292 pgs: 292 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:16.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:15:16.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4255550322' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi006-81775-40"}]': finished 2024-03-24T12:15:16.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-33", "mode": "writeback"}]': finished 2024-03-24T12:15:16.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi006-81794-50"}]': finished 2024-03-24T12:15:16.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[41734]: osdmap e260: 8 total, 8 up, 8 in 2024-03-24T12:15:16.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi006-81794-50"}]: dispatch 2024-03-24T12:15:16.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1291879078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:16.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:16.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1291879078' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:16.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:16.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1291879078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi006-81775-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:16.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi006-81775-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:16.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:16.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:16.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:15 smithi006 ceph-mon[41734]: pgmap v348: 292 pgs: 292 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:17.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi006-81794-50"}]': finished 2024-03-24T12:15:17.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi006-81775-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:17.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:15:17.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-33"}]: dispatch 2024-03-24T12:15:17.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:17 smithi090 ceph-mon[33646]: osdmap e261: 8 total, 8 up, 8 in 2024-03-24T12:15:17.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-33"}]: dispatch 2024-03-24T12:15:17.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1291879078' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi006-81775-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:17.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi006-81775-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:17.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi006-81794-50"}]': finished 2024-03-24T12:15:17.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi006-81775-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:17.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:15:17.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-33"}]: dispatch 2024-03-24T12:15:17.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:17 smithi006 ceph-mon[30108]: osdmap e261: 8 total, 8 up, 8 in 2024-03-24T12:15:17.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-33"}]: dispatch 2024-03-24T12:15:17.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1291879078' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi006-81775-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:17.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi006-81775-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:17.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3205418731' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi006-81794-50"}]': finished 2024-03-24T12:15:17.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi006-81775-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:17.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:15:17.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-33"}]: dispatch 2024-03-24T12:15:17.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:17 smithi006 ceph-mon[41734]: osdmap e261: 8 total, 8 up, 8 in 2024-03-24T12:15:17.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-33"}]: dispatch 2024-03-24T12:15:17.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1291879078' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi006-81775-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:17.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi006-81775-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:18.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:18 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:15:18.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-33"}]': finished 2024-03-24T12:15:18.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:18 smithi090 ceph-mon[33646]: pgmap v351: 324 pgs: 10 creating+peering, 22 unknown, 292 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:15:18.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:18 smithi090 ceph-mon[33646]: osdmap e262: 8 total, 8 up, 8 in 2024-03-24T12:15:18.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2801914631' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi006-81794-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:18.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi006-81794-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:18.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:18 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:15:18.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:18 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:18.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi006-81775-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi006-81775-41"}]': finished 2024-03-24T12:15:18.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi006-81794-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:18.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:18 smithi090 ceph-mon[33646]: osdmap e263: 8 total, 8 up, 8 in 2024-03-24T12:15:18.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:18 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:15:18.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-33"}]': finished 2024-03-24T12:15:18.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:18 smithi006 ceph-mon[30108]: pgmap v351: 324 pgs: 10 creating+peering, 22 unknown, 292 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:15:18.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:18 smithi006 ceph-mon[30108]: osdmap e262: 8 total, 8 up, 8 in 2024-03-24T12:15:18.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2801914631' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi006-81794-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:18.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi006-81794-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:18.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:18 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:15:18.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:18 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:18.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi006-81775-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi006-81775-41"}]': finished 2024-03-24T12:15:18.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi006-81794-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:18.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:18 smithi006 ceph-mon[30108]: osdmap e263: 8 total, 8 up, 8 in 2024-03-24T12:15:18.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:18 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:15:18.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-33"}]': finished 2024-03-24T12:15:18.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:18 smithi006 ceph-mon[41734]: pgmap v351: 324 pgs: 10 creating+peering, 22 unknown, 292 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:15:18.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:18 smithi006 ceph-mon[41734]: osdmap e262: 8 total, 8 up, 8 in 2024-03-24T12:15:18.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2801914631' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi006-81794-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:18.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi006-81794-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:18.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:18 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:15:18.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:18 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:18.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi006-81775-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi006-81775-41"}]': finished 2024-03-24T12:15:18.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi006-81794-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:18.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:18 smithi006 ceph-mon[41734]: osdmap e263: 8 total, 8 up, 8 in 2024-03-24T12:15:20.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:19 smithi090 ceph-mon[33646]: osdmap e264: 8 total, 8 up, 8 in 2024-03-24T12:15:20.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:20.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:20.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi006-81794-52"}]: dispatch 2024-03-24T12:15:20.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi006-81794-52"}]: dispatch 2024-03-24T12:15:20.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi006-81794-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:20.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:19 smithi090 ceph-mon[33646]: pgmap v354: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:15:20.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:19 smithi006 ceph-mon[30108]: osdmap e264: 8 total, 8 up, 8 in 2024-03-24T12:15:20.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:20.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:20.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi006-81794-52"}]: dispatch 2024-03-24T12:15:20.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi006-81794-52"}]: dispatch 2024-03-24T12:15:20.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi006-81794-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:20.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:19 smithi006 ceph-mon[30108]: pgmap v354: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:15:20.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:19 smithi006 ceph-mon[41734]: osdmap e264: 8 total, 8 up, 8 in 2024-03-24T12:15:20.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:20.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:20.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi006-81794-52"}]: dispatch 2024-03-24T12:15:20.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi006-81794-52"}]: dispatch 2024-03-24T12:15:20.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi006-81794-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:20.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:19 smithi006 ceph-mon[41734]: pgmap v354: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:15:21.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:21 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:21.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi006-81794-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:21.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:21 smithi090 ceph-mon[33646]: osdmap e265: 8 total, 8 up, 8 in 2024-03-24T12:15:21.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:21.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi006-81794-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi006-81794-52"}]: dispatch 2024-03-24T12:15:21.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:21 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:21.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1291879078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:21.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:21 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:21.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:21 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:21.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi006-81794-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:21.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:21 smithi006 ceph-mon[30108]: osdmap e265: 8 total, 8 up, 8 in 2024-03-24T12:15:21.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:21.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi006-81794-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi006-81794-52"}]: dispatch 2024-03-24T12:15:21.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:21 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:21.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1291879078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:21.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:21 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:21.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:21 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:21.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi006-81794-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:21.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:21 smithi006 ceph-mon[41734]: osdmap e265: 8 total, 8 up, 8 in 2024-03-24T12:15:21.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:21.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi006-81794-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi006-81794-52"}]: dispatch 2024-03-24T12:15:21.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:21 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:21.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1291879078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:21.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:21 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:22.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:15:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:15:22.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-35", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:15:22.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi006-81775-41"}]': finished 2024-03-24T12:15:22.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1291879078' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:22.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-35"}]: dispatch 2024-03-24T12:15:22.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:22 smithi090 ceph-mon[33646]: pgmap v357: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:15:22.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:22 smithi090 ceph-mon[33646]: osdmap e266: 8 total, 8 up, 8 in 2024-03-24T12:15:22.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-35"}]: dispatch 2024-03-24T12:15:22.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:22.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi006-81794-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi006-81794-52"}]': finished 2024-03-24T12:15:22.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-35"}]': finished 2024-03-24T12:15:22.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi006-81775-41"}]': finished 2024-03-24T12:15:22.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-35", "mode": "writeback"}]: dispatch 2024-03-24T12:15:22.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:22 smithi090 ceph-mon[33646]: osdmap e267: 8 total, 8 up, 8 in 2024-03-24T12:15:22.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-35", "mode": "writeback"}]: dispatch 2024-03-24T12:15:22.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi006-81775-42"}]: dispatch 2024-03-24T12:15:22.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi006-81775-42"}]: dispatch 2024-03-24T12:15:22.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi006-81775-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:22.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-35", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:15:22.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi006-81775-41"}]': finished 2024-03-24T12:15:22.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1291879078' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:22.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-35"}]: dispatch 2024-03-24T12:15:22.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[30108]: pgmap v357: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:15:22.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[30108]: osdmap e266: 8 total, 8 up, 8 in 2024-03-24T12:15:22.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-35"}]: dispatch 2024-03-24T12:15:22.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:22.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi006-81794-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi006-81794-52"}]': finished 2024-03-24T12:15:22.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-35"}]': finished 2024-03-24T12:15:22.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi006-81775-41"}]': finished 2024-03-24T12:15:22.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-35", "mode": "writeback"}]: dispatch 2024-03-24T12:15:22.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[30108]: osdmap e267: 8 total, 8 up, 8 in 2024-03-24T12:15:22.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-35", "mode": "writeback"}]: dispatch 2024-03-24T12:15:22.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi006-81775-42"}]: dispatch 2024-03-24T12:15:22.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi006-81775-42"}]: dispatch 2024-03-24T12:15:22.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi006-81775-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:22.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-35", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:15:22.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi006-81775-41"}]': finished 2024-03-24T12:15:22.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1291879078' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:22.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-35"}]: dispatch 2024-03-24T12:15:22.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[41734]: pgmap v357: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:15:22.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[41734]: osdmap e266: 8 total, 8 up, 8 in 2024-03-24T12:15:22.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-35"}]: dispatch 2024-03-24T12:15:22.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi006-81775-41"}]: dispatch 2024-03-24T12:15:22.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi006-81794-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi006-81794-52"}]': finished 2024-03-24T12:15:22.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-35"}]': finished 2024-03-24T12:15:22.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi006-81775-41"}]': finished 2024-03-24T12:15:22.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-35", "mode": "writeback"}]: dispatch 2024-03-24T12:15:22.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[41734]: osdmap e267: 8 total, 8 up, 8 in 2024-03-24T12:15:22.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-35", "mode": "writeback"}]: dispatch 2024-03-24T12:15:22.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi006-81775-42"}]: dispatch 2024-03-24T12:15:22.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi006-81775-42"}]: dispatch 2024-03-24T12:15:22.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi006-81775-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:23.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:23 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:15:23.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:23 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:15:23.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-35", "mode": "writeback"}]': finished 2024-03-24T12:15:23.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi006-81775-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:23.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:23 smithi090 ceph-mon[33646]: osdmap e268: 8 total, 8 up, 8 in 2024-03-24T12:15:23.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi006-81775-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi006-81775-42"}]: dispatch 2024-03-24T12:15:23.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:23 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:15:23.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:23 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:15:23.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-35", "mode": "writeback"}]': finished 2024-03-24T12:15:23.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi006-81775-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:23.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:23 smithi006 ceph-mon[30108]: osdmap e268: 8 total, 8 up, 8 in 2024-03-24T12:15:23.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi006-81775-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi006-81775-42"}]: dispatch 2024-03-24T12:15:23.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:23 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:15:23.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:23 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:15:23.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-35", "mode": "writeback"}]': finished 2024-03-24T12:15:23.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi006-81775-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:23.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:23 smithi006 ceph-mon[41734]: osdmap e268: 8 total, 8 up, 8 in 2024-03-24T12:15:23.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi006-81775-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi006-81775-42"}]: dispatch 2024-03-24T12:15:24.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:24 smithi006 ceph-mon[30108]: pgmap v360: 300 pgs: 5 creating+peering, 3 unknown, 292 active+clean; 4.6 MiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:24.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:24 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:24.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:24 smithi006 ceph-mon[41734]: pgmap v360: 300 pgs: 5 creating+peering, 3 unknown, 292 active+clean; 4.6 MiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:24.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:24 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:24.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:24 smithi090 ceph-mon[33646]: pgmap v360: 300 pgs: 5 creating+peering, 3 unknown, 292 active+clean; 4.6 MiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:24.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:24 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:25.461 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:15:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:15:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:15:25.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:25 smithi006 ceph-mon[30108]: osdmap e269: 8 total, 8 up, 8 in 2024-03-24T12:15:25.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi006-81794-52"}]: dispatch 2024-03-24T12:15:25.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi006-81775-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi006-81775-42"}]': finished 2024-03-24T12:15:25.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi006-81794-52"}]': finished 2024-03-24T12:15:25.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:25 smithi006 ceph-mon[30108]: osdmap e270: 8 total, 8 up, 8 in 2024-03-24T12:15:25.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi006-81794-52"}]: dispatch 2024-03-24T12:15:25.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:25.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:25 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:25.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:25 smithi006 ceph-mon[41734]: osdmap e269: 8 total, 8 up, 8 in 2024-03-24T12:15:25.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi006-81794-52"}]: dispatch 2024-03-24T12:15:25.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi006-81775-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi006-81775-42"}]': finished 2024-03-24T12:15:25.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi006-81794-52"}]': finished 2024-03-24T12:15:25.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:25 smithi006 ceph-mon[41734]: osdmap e270: 8 total, 8 up, 8 in 2024-03-24T12:15:25.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi006-81794-52"}]: dispatch 2024-03-24T12:15:25.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:25.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:25 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:25.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:25 smithi090 ceph-mon[33646]: osdmap e269: 8 total, 8 up, 8 in 2024-03-24T12:15:25.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi006-81794-52"}]: dispatch 2024-03-24T12:15:25.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi006-81775-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi006-81775-42"}]': finished 2024-03-24T12:15:25.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi006-81794-52"}]': finished 2024-03-24T12:15:25.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:25 smithi090 ceph-mon[33646]: osdmap e270: 8 total, 8 up, 8 in 2024-03-24T12:15:25.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi006-81794-52"}]: dispatch 2024-03-24T12:15:25.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:25.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:25 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[30108]: pgmap v363: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:26.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi006-81794-52"}]': finished 2024-03-24T12:15:26.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:15:26.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[30108]: osdmap e271: 8 total, 8 up, 8 in 2024-03-24T12:15:26.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-35"}]: dispatch 2024-03-24T12:15:26.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-35"}]: dispatch 2024-03-24T12:15:26.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2673928867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:26.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:26.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2673928867' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:26.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:26.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2673928867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:26.463 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:26.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[41734]: pgmap v363: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:26.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi006-81794-52"}]': finished 2024-03-24T12:15:26.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:15:26.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[41734]: osdmap e271: 8 total, 8 up, 8 in 2024-03-24T12:15:26.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-35"}]: dispatch 2024-03-24T12:15:26.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-35"}]: dispatch 2024-03-24T12:15:26.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2673928867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:26.464 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:26.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2673928867' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:26.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:26.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2673928867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:26.465 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:26.580 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:26 smithi090 ceph-mon[33646]: pgmap v363: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:26.580 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/294250954' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi006-81794-52"}]': finished 2024-03-24T12:15:26.580 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:15:26.580 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:26 smithi090 ceph-mon[33646]: osdmap e271: 8 total, 8 up, 8 in 2024-03-24T12:15:26.580 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-35"}]: dispatch 2024-03-24T12:15:26.580 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-35"}]: dispatch 2024-03-24T12:15:26.580 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2673928867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:26.580 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:26.580 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2673928867' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:26.581 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:26.581 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2673928867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:26.581 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:28.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:27 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-35"}]': finished 2024-03-24T12:15:28.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:27 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:28.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:27 smithi006 ceph-mon[30108]: osdmap e272: 8 total, 8 up, 8 in 2024-03-24T12:15:28.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:27 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi006-81775-42"}]: dispatch 2024-03-24T12:15:28.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:27 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2673928867' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi006-81794-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:28.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:27 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi006-81794-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:28.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:27 smithi006 ceph-mon[30108]: pgmap v366: 292 pgs: 292 active+clean; 4.6 MiB data, 587 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 767 B/s wr, 1 op/s 2024-03-24T12:15:28.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:27 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-35"}]': finished 2024-03-24T12:15:28.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:27 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:28.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:27 smithi006 ceph-mon[41734]: osdmap e272: 8 total, 8 up, 8 in 2024-03-24T12:15:28.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:27 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi006-81775-42"}]: dispatch 2024-03-24T12:15:28.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:27 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2673928867' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi006-81794-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:28.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:27 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi006-81794-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:28.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:27 smithi006 ceph-mon[41734]: pgmap v366: 292 pgs: 292 active+clean; 4.6 MiB data, 587 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 767 B/s wr, 1 op/s 2024-03-24T12:15:28.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:27 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-35"}]': finished 2024-03-24T12:15:28.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:27 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:28.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:27 smithi090 ceph-mon[33646]: osdmap e272: 8 total, 8 up, 8 in 2024-03-24T12:15:28.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:27 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi006-81775-42"}]: dispatch 2024-03-24T12:15:28.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:27 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2673928867' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi006-81794-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:28.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:27 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi006-81794-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:28.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:27 smithi090 ceph-mon[33646]: pgmap v366: 292 pgs: 292 active+clean; 4.6 MiB data, 587 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 767 B/s wr, 1 op/s 2024-03-24T12:15:28.952 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: sta api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7041 ms) 2024-03-24T12:15:28.952 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2024-03-24T12:15:28.953 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (6895 ms) 2024-03-24T12:15:28.953 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-03-24T12:15:28.953 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (7033 ms) 2024-03-24T12:15:28.953 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-03-24T12:15:28.953 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (6984 ms) 2024-03-24T12:15:28.953 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-03-24T12:15:28.953 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7031 ms) 2024-03-24T12:15:28.953 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-03-24T12:15:28.953 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7043 ms) 2024-03-24T12:15:28.953 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-03-24T12:15:28.953 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7056 ms) 2024-03-24T12:15:28.953 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-03-24T12:15:28.953 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7030 ms) 2024-03-24T12:15:28.953 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-03-24T12:15:28.954 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (6853 ms) 2024-03-24T12:15:28.954 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-03-24T12:15:28.954 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7028 ms) 2024-03-24T12:15:28.954 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-03-24T12:15:28.954 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (6960 ms) 2024-03-24T12:15:28.954 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-03-24T12:15:28.954 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (7023 ms) 2024-03-24T12:15:28.954 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-03-24T12:15:28.954 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (6981 ms) 2024-03-24T12:15:28.954 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-03-24T12:15:28.954 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7034 ms) 2024-03-24T12:15:28.954 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-03-24T12:15:28.954 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7031 ms) 2024-03-24T12:15:28.954 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-03-24T12:15:28.955 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (7035 ms) 2024-03-24T12:15:28.955 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (112058 ms total) 2024-03-24T12:15:28.955 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: 2024-03-24T12:15:28.955 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [----------] Global test environment tear-down 2024-03-24T12:15:28.955 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (207589 ms total) 2024-03-24T12:15:28.955 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio: [ PASSED ] 42 tests. 2024-03-24T12:15:29.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi006-81775-42"}]': finished 2024-03-24T12:15:29.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:28 smithi006 ceph-mon[30108]: osdmap e273: 8 total, 8 up, 8 in 2024-03-24T12:15:29.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi006-81775-42"}]: dispatch 2024-03-24T12:15:29.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:29.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:29.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi006-81775-42"}]': finished 2024-03-24T12:15:29.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:28 smithi006 ceph-mon[41734]: osdmap e273: 8 total, 8 up, 8 in 2024-03-24T12:15:29.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi006-81775-42"}]: dispatch 2024-03-24T12:15:29.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:29.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:29.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi006-81775-42"}]': finished 2024-03-24T12:15:29.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:28 smithi090 ceph-mon[33646]: osdmap e273: 8 total, 8 up, 8 in 2024-03-24T12:15:29.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi006-81775-42"}]: dispatch 2024-03-24T12:15:29.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:29.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:30.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi006-81794-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi006-81794-53"}]': finished 2024-03-24T12:15:30.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:29 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi006-81775-42"}]': finished 2024-03-24T12:15:30.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:15:30.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:29 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-35"}]: dispatch 2024-03-24T12:15:30.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:29 smithi006 ceph-mon[30108]: osdmap e274: 8 total, 8 up, 8 in 2024-03-24T12:15:30.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-35"}]: dispatch 2024-03-24T12:15:30.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:29 smithi006 ceph-mon[30108]: pgmap v369: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-24T12:15:30.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:29 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:15:30.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-35"}]': finished 2024-03-24T12:15:30.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:29 smithi006 ceph-mon[30108]: osdmap e275: 8 total, 8 up, 8 in 2024-03-24T12:15:30.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi006-81794-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi006-81794-53"}]': finished 2024-03-24T12:15:30.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:29 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi006-81775-42"}]': finished 2024-03-24T12:15:30.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:15:30.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:29 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-35"}]: dispatch 2024-03-24T12:15:30.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:29 smithi006 ceph-mon[41734]: osdmap e274: 8 total, 8 up, 8 in 2024-03-24T12:15:30.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-35"}]: dispatch 2024-03-24T12:15:30.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:29 smithi006 ceph-mon[41734]: pgmap v369: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-24T12:15:30.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:29 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:15:30.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-35"}]': finished 2024-03-24T12:15:30.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:29 smithi006 ceph-mon[41734]: osdmap e275: 8 total, 8 up, 8 in 2024-03-24T12:15:30.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi006-81794-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi006-81794-53"}]': finished 2024-03-24T12:15:30.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:29 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1794674028' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi006-81775-42"}]': finished 2024-03-24T12:15:30.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:15:30.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:29 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-35"}]: dispatch 2024-03-24T12:15:30.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:29 smithi090 ceph-mon[33646]: osdmap e274: 8 total, 8 up, 8 in 2024-03-24T12:15:30.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-35"}]: dispatch 2024-03-24T12:15:30.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:29 smithi090 ceph-mon[33646]: pgmap v369: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-24T12:15:30.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:29 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:15:30.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-35"}]': finished 2024-03-24T12:15:30.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:29 smithi090 ceph-mon[33646]: osdmap e275: 8 total, 8 up, 8 in 2024-03-24T12:15:31.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:31 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:31.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:31 smithi006 ceph-mon[30108]: osdmap e276: 8 total, 8 up, 8 in 2024-03-24T12:15:31.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2673928867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:31.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:31 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:31.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:31 smithi006 ceph-mon[41734]: osdmap e276: 8 total, 8 up, 8 in 2024-03-24T12:15:31.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2673928867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:31.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:31 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:31.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:31 smithi090 ceph-mon[33646]: osdmap e276: 8 total, 8 up, 8 in 2024-03-24T12:15:31.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2673928867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:32 smithi006 ceph-mon[30108]: pgmap v372: 260 pgs: 260 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:32 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:32 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53"}]': finished 2024-03-24T12:15:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:32 smithi006 ceph-mon[30108]: osdmap e277: 8 total, 8 up, 8 in 2024-03-24T12:15:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:32 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2673928867' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:32 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:32 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:32 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:32.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:32 smithi006 ceph-mon[41734]: pgmap v372: 260 pgs: 260 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:32.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:32 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:32.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:32 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53"}]': finished 2024-03-24T12:15:32.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:32 smithi006 ceph-mon[41734]: osdmap e277: 8 total, 8 up, 8 in 2024-03-24T12:15:32.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:32 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2673928867' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:32.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:32 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:32.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:32 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:32.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:32 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:32.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:15:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:15:32.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:32 smithi090 ceph-mon[33646]: pgmap v372: 260 pgs: 260 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:32.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:32 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:32.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:32 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi006-81794-53"}]': finished 2024-03-24T12:15:32.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:32 smithi090 ceph-mon[33646]: osdmap e277: 8 total, 8 up, 8 in 2024-03-24T12:15:32.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:32 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2673928867' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:32.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:32 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi006-81794-53"}]: dispatch 2024-03-24T12:15:32.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:32 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:32.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:32 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:33 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:15:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:33 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:15:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:33 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi006-81794-53"}]': finished 2024-03-24T12:15:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:33 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:33 smithi006 ceph-mon[30108]: osdmap e278: 8 total, 8 up, 8 in 2024-03-24T12:15:33.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:33.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:33 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:33.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi006-81794-54"}]: dispatch 2024-03-24T12:15:33.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi006-81794-54"}]: dispatch 2024-03-24T12:15:33.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi006-81794-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:33.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:33 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:15:33.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:33 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:15:33.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:33 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi006-81794-53"}]': finished 2024-03-24T12:15:33.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:33 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:33.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:33 smithi006 ceph-mon[41734]: osdmap e278: 8 total, 8 up, 8 in 2024-03-24T12:15:33.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:33.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:33 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:33.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi006-81794-54"}]: dispatch 2024-03-24T12:15:33.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi006-81794-54"}]: dispatch 2024-03-24T12:15:33.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi006-81794-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:33.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:33 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:15:33.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:33 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:15:33.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:33 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi006-81794-53"}]': finished 2024-03-24T12:15:33.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:33 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:33.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:33 smithi090 ceph-mon[33646]: osdmap e278: 8 total, 8 up, 8 in 2024-03-24T12:15:33.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:33.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:33 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:33.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi006-81794-54"}]: dispatch 2024-03-24T12:15:33.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi006-81794-54"}]: dispatch 2024-03-24T12:15:33.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi006-81794-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:34.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:34 smithi006 ceph-mon[30108]: pgmap v375: 292 pgs: 32 creating+peering, 260 active+clean; 4.6 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-24T12:15:34.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:34 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-37", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:15:34.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi006-81794-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:34.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-37"}]: dispatch 2024-03-24T12:15:34.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:34 smithi006 ceph-mon[30108]: osdmap e279: 8 total, 8 up, 8 in 2024-03-24T12:15:34.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi006-81794-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi006-81794-54"}]: dispatch 2024-03-24T12:15:34.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:34 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-37"}]: dispatch 2024-03-24T12:15:34.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:34 smithi006 ceph-mon[41734]: pgmap v375: 292 pgs: 32 creating+peering, 260 active+clean; 4.6 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-24T12:15:34.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:34 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-37", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:15:34.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi006-81794-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:34.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-37"}]: dispatch 2024-03-24T12:15:34.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:34 smithi006 ceph-mon[41734]: osdmap e279: 8 total, 8 up, 8 in 2024-03-24T12:15:34.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi006-81794-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi006-81794-54"}]: dispatch 2024-03-24T12:15:34.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:34 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-37"}]: dispatch 2024-03-24T12:15:34.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:34 smithi090 ceph-mon[33646]: pgmap v375: 292 pgs: 32 creating+peering, 260 active+clean; 4.6 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-24T12:15:34.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:34 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-37", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:15:34.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi006-81794-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:34.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-37"}]: dispatch 2024-03-24T12:15:34.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:34 smithi090 ceph-mon[33646]: osdmap e279: 8 total, 8 up, 8 in 2024-03-24T12:15:34.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi006-81794-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi006-81794-54"}]: dispatch 2024-03-24T12:15:34.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:34 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-37"}]: dispatch 2024-03-24T12:15:35.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:15:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:15:36.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-37"}]': finished 2024-03-24T12:15:36.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:35 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-37", "mode": "writeback"}]: dispatch 2024-03-24T12:15:36.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:35 smithi090 ceph-mon[33646]: pgmap v378: 292 pgs: 32 creating+peering, 260 active+clean; 4.6 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:36.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:35 smithi090 ceph-mon[33646]: osdmap e280: 8 total, 8 up, 8 in 2024-03-24T12:15:36.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-37", "mode": "writeback"}]: dispatch 2024-03-24T12:15:36.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:35 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:36.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:35 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:15:36.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:35 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi006-81794-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi006-81794-54"}]': finished 2024-03-24T12:15:36.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-37", "mode": "writeback"}]': finished 2024-03-24T12:15:36.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:35 smithi090 ceph-mon[33646]: osdmap e281: 8 total, 8 up, 8 in 2024-03-24T12:15:36.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:35 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:36.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:36.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-37"}]': finished 2024-03-24T12:15:36.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-37", "mode": "writeback"}]: dispatch 2024-03-24T12:15:36.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[30108]: pgmap v378: 292 pgs: 32 creating+peering, 260 active+clean; 4.6 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:36.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[30108]: osdmap e280: 8 total, 8 up, 8 in 2024-03-24T12:15:36.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-37", "mode": "writeback"}]: dispatch 2024-03-24T12:15:36.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:36.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:15:36.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi006-81794-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi006-81794-54"}]': finished 2024-03-24T12:15:36.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-37", "mode": "writeback"}]': finished 2024-03-24T12:15:36.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[30108]: osdmap e281: 8 total, 8 up, 8 in 2024-03-24T12:15:36.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:36.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:36.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-37"}]': finished 2024-03-24T12:15:36.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-37", "mode": "writeback"}]: dispatch 2024-03-24T12:15:36.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[41734]: pgmap v378: 292 pgs: 32 creating+peering, 260 active+clean; 4.6 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:36.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[41734]: osdmap e280: 8 total, 8 up, 8 in 2024-03-24T12:15:36.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-37", "mode": "writeback"}]: dispatch 2024-03-24T12:15:36.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:36.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:15:36.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi006-81794-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi006-81794-54"}]': finished 2024-03-24T12:15:36.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-37", "mode": "writeback"}]': finished 2024-03-24T12:15:36.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[41734]: osdmap e281: 8 total, 8 up, 8 in 2024-03-24T12:15:36.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:36.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:37 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:15:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:37 smithi090 ceph-mon[33646]: osdmap e282: 8 total, 8 up, 8 in 2024-03-24T12:15:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-37"}]: dispatch 2024-03-24T12:15:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:37 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-37"}]: dispatch 2024-03-24T12:15:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:37 smithi090 ceph-mon[33646]: pgmap v381: 300 pgs: 2 unknown, 7 creating+peering, 291 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-03-24T12:15:38.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:37 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:15:38.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:37 smithi006 ceph-mon[30108]: osdmap e282: 8 total, 8 up, 8 in 2024-03-24T12:15:38.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-37"}]: dispatch 2024-03-24T12:15:38.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:37 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-37"}]: dispatch 2024-03-24T12:15:38.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:37 smithi006 ceph-mon[30108]: pgmap v381: 300 pgs: 2 unknown, 7 creating+peering, 291 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-03-24T12:15:38.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:37 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:15:38.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:37 smithi006 ceph-mon[41734]: osdmap e282: 8 total, 8 up, 8 in 2024-03-24T12:15:38.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-37"}]: dispatch 2024-03-24T12:15:38.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:37 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-37"}]: dispatch 2024-03-24T12:15:38.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:37 smithi006 ceph-mon[41734]: pgmap v381: 300 pgs: 2 unknown, 7 creating+peering, 291 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-03-24T12:15:39.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:38 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:15:39.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-37"}]': finished 2024-03-24T12:15:39.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:38 smithi090 ceph-mon[33646]: osdmap e283: 8 total, 8 up, 8 in 2024-03-24T12:15:39.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:38 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi006-81794-54"}]: dispatch 2024-03-24T12:15:39.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:38 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:15:39.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-37"}]': finished 2024-03-24T12:15:39.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:38 smithi006 ceph-mon[30108]: osdmap e283: 8 total, 8 up, 8 in 2024-03-24T12:15:39.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:38 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi006-81794-54"}]: dispatch 2024-03-24T12:15:39.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:38 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:15:39.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-37"}]': finished 2024-03-24T12:15:39.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:38 smithi006 ceph-mon[41734]: osdmap e283: 8 total, 8 up, 8 in 2024-03-24T12:15:39.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:38 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi006-81794-54"}]: dispatch 2024-03-24T12:15:40.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:39 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi006-81794-54"}]': finished 2024-03-24T12:15:40.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:39 smithi090 ceph-mon[33646]: osdmap e284: 8 total, 8 up, 8 in 2024-03-24T12:15:40.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:39 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi006-81794-54"}]: dispatch 2024-03-24T12:15:40.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:39 smithi090 ceph-mon[33646]: pgmap v384: 260 pgs: 260 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-24T12:15:40.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi006-81794-54"}]': finished 2024-03-24T12:15:40.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:39 smithi006 ceph-mon[30108]: osdmap e284: 8 total, 8 up, 8 in 2024-03-24T12:15:40.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi006-81794-54"}]: dispatch 2024-03-24T12:15:40.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:39 smithi006 ceph-mon[30108]: pgmap v384: 260 pgs: 260 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-24T12:15:40.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:39 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi006-81794-54"}]': finished 2024-03-24T12:15:40.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:39 smithi006 ceph-mon[41734]: osdmap e284: 8 total, 8 up, 8 in 2024-03-24T12:15:40.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:39 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi006-81794-54"}]: dispatch 2024-03-24T12:15:40.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:39 smithi006 ceph-mon[41734]: pgmap v384: 260 pgs: 260 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-24T12:15:41.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi006-81794-54"}]': finished 2024-03-24T12:15:41.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:40 smithi090 ceph-mon[33646]: osdmap e285: 8 total, 8 up, 8 in 2024-03-24T12:15:41.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:41.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:40 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:41.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi006-81794-55"}]: dispatch 2024-03-24T12:15:41.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi006-81794-55"}]: dispatch 2024-03-24T12:15:41.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi006-81794-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:41.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi006-81794-54"}]': finished 2024-03-24T12:15:41.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:40 smithi006 ceph-mon[30108]: osdmap e285: 8 total, 8 up, 8 in 2024-03-24T12:15:41.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:41.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:40 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:41.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi006-81794-55"}]: dispatch 2024-03-24T12:15:41.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi006-81794-55"}]: dispatch 2024-03-24T12:15:41.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi006-81794-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:41.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3188579890' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi006-81794-54"}]': finished 2024-03-24T12:15:41.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:40 smithi006 ceph-mon[41734]: osdmap e285: 8 total, 8 up, 8 in 2024-03-24T12:15:41.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:41.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:40 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:41.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi006-81794-55"}]: dispatch 2024-03-24T12:15:41.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi006-81794-55"}]: dispatch 2024-03-24T12:15:41.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi006-81794-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:42.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:15:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:15:42.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:42 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:42.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:42 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi006-81794-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:42.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:42 smithi090 ceph-mon[33646]: osdmap e286: 8 total, 8 up, 8 in 2024-03-24T12:15:42.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:42 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi006-81794-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi006-81794-55"}]: dispatch 2024-03-24T12:15:42.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:42 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:42.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:42 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:42.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:42 smithi090 ceph-mon[33646]: pgmap v387: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:15:42.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:42 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-39", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:15:42.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:42 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:42.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:42 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi006-81794-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:42.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:42 smithi006 ceph-mon[41734]: osdmap e286: 8 total, 8 up, 8 in 2024-03-24T12:15:42.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:42 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi006-81794-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi006-81794-55"}]: dispatch 2024-03-24T12:15:42.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:42 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:42.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:42 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:42.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:42 smithi006 ceph-mon[41734]: pgmap v387: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:15:42.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:42 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-39", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:15:42.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:42 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:42.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:42 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi006-81794-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:42.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:42 smithi006 ceph-mon[30108]: osdmap e286: 8 total, 8 up, 8 in 2024-03-24T12:15:42.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:42 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi006-81794-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi006-81794-55"}]: dispatch 2024-03-24T12:15:42.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:42 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:42.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:42 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:42.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:42 smithi006 ceph-mon[30108]: pgmap v387: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:15:42.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:42 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-39", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:15:43.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:43 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-39"}]: dispatch 2024-03-24T12:15:43.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:43 smithi090 ceph-mon[33646]: osdmap e287: 8 total, 8 up, 8 in 2024-03-24T12:15:43.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:43 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-39"}]: dispatch 2024-03-24T12:15:43.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:43 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:15:43.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:43 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi006-81794-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi006-81794-55"}]': finished 2024-03-24T12:15:43.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:43 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-39"}]': finished 2024-03-24T12:15:43.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:43 smithi090 ceph-mon[33646]: osdmap e288: 8 total, 8 up, 8 in 2024-03-24T12:15:43.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:43 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-39", "mode": "writeback"}]: dispatch 2024-03-24T12:15:43.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:43 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-39", "mode": "writeback"}]: dispatch 2024-03-24T12:15:43.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:43 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-39"}]: dispatch 2024-03-24T12:15:43.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:43 smithi006 ceph-mon[30108]: osdmap e287: 8 total, 8 up, 8 in 2024-03-24T12:15:43.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:43 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-39"}]: dispatch 2024-03-24T12:15:43.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:43 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:15:43.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:43 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi006-81794-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi006-81794-55"}]': finished 2024-03-24T12:15:43.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:43 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-39"}]': finished 2024-03-24T12:15:43.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:43 smithi006 ceph-mon[30108]: osdmap e288: 8 total, 8 up, 8 in 2024-03-24T12:15:43.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:43 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-39", "mode": "writeback"}]: dispatch 2024-03-24T12:15:43.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:43 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-39", "mode": "writeback"}]: dispatch 2024-03-24T12:15:43.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:43 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-39"}]: dispatch 2024-03-24T12:15:43.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:43 smithi006 ceph-mon[41734]: osdmap e287: 8 total, 8 up, 8 in 2024-03-24T12:15:43.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:43 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-39"}]: dispatch 2024-03-24T12:15:43.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:43 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:15:43.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:43 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi006-81794-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi006-81794-55"}]': finished 2024-03-24T12:15:43.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:43 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-39"}]': finished 2024-03-24T12:15:43.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:43 smithi006 ceph-mon[41734]: osdmap e288: 8 total, 8 up, 8 in 2024-03-24T12:15:43.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:43 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-39", "mode": "writeback"}]: dispatch 2024-03-24T12:15:43.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:43 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-39", "mode": "writeback"}]: dispatch 2024-03-24T12:15:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:44 smithi090 ceph-mon[33646]: pgmap v390: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:44 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:44 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:15:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:44 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-39", "mode": "writeback"}]': finished 2024-03-24T12:15:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:44 smithi090 ceph-mon[33646]: osdmap e289: 8 total, 8 up, 8 in 2024-03-24T12:15:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:44 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:44 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:44.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:44 smithi006 ceph-mon[30108]: pgmap v390: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:44.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:44 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:44.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:44 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:15:44.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:44 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-39", "mode": "writeback"}]': finished 2024-03-24T12:15:44.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:44 smithi006 ceph-mon[30108]: osdmap e289: 8 total, 8 up, 8 in 2024-03-24T12:15:44.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:44 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:44.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:44 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:44.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:44 smithi006 ceph-mon[41734]: pgmap v390: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:44.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:44 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:44.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:44 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:15:44.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:44 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-39", "mode": "writeback"}]': finished 2024-03-24T12:15:44.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:44 smithi006 ceph-mon[41734]: osdmap e289: 8 total, 8 up, 8 in 2024-03-24T12:15:44.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:44 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:44.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:44 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:45.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:15:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:15:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:15:46.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:15:46.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:45 smithi090 ceph-mon[33646]: osdmap e290: 8 total, 8 up, 8 in 2024-03-24T12:15:46.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-39"}]: dispatch 2024-03-24T12:15:46.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-39"}]: dispatch 2024-03-24T12:15:46.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi006-81794-55"}]: dispatch 2024-03-24T12:15:46.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:45 smithi090 ceph-mon[33646]: pgmap v393: 292 pgs: 292 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:46.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:15:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:45 smithi006 ceph-mon[30108]: osdmap e290: 8 total, 8 up, 8 in 2024-03-24T12:15:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-39"}]: dispatch 2024-03-24T12:15:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-39"}]: dispatch 2024-03-24T12:15:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi006-81794-55"}]: dispatch 2024-03-24T12:15:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:45 smithi006 ceph-mon[30108]: pgmap v393: 292 pgs: 292 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:46.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:15:46.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:45 smithi006 ceph-mon[41734]: osdmap e290: 8 total, 8 up, 8 in 2024-03-24T12:15:46.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-39"}]: dispatch 2024-03-24T12:15:46.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-39"}]: dispatch 2024-03-24T12:15:46.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi006-81794-55"}]: dispatch 2024-03-24T12:15:46.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:45 smithi006 ceph-mon[41734]: pgmap v393: 292 pgs: 292 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:46.892 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-03-24T12:15:46.892 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-03-24T12:15:46.892 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-03-24T12:15:46.892 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: seed 82061 2024-03-24T12:15:46.892 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-03-24T12:15:46.892 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (357 ms) 2024-03-24T12:15:46.892 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-03-24T12:15:46.892 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (18592 ms) 2024-03-24T12:15:46.892 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-03-24T12:15:46.893 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (38 ms) 2024-03-24T12:15:46.893 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (18987 ms total) 2024-03-24T12:15:46.893 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: 2024-03-24T12:15:46.893 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-03-24T12:15:46.893 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-03-24T12:15:46.893 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (7012 ms) 2024-03-24T12:15:46.893 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-03-24T12:15:46.893 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (8003 ms) 2024-03-24T12:15:46.893 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-03-24T12:15:46.893 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (9993 ms) 2024-03-24T12:15:46.893 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-03-24T12:15:46.893 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: my_snaps [3] 2024-03-24T12:15:46.893 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: my_snaps [4,3] 2024-03-24T12:15:46.893 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: my_snaps [5,4,3] 2024-03-24T12:15:46.893 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: my_snaps [6,5,4,3] 2024-03-24T12:15:46.894 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: promoting some heads 2024-03-24T12:15:46.894 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: promoting from clones for snap 6 2024-03-24T12:15:46.894 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: promoting from clones for snap 5 2024-03-24T12:15:46.894 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: promoting from clones for snap 4 2024-03-24T12:15:46.894 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: promoting from clones for snap 3 2024-03-24T12:15:46.894 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: waiting for scrubs... 2024-03-24T12:15:46.894 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: done waiting 2024-03-24T12:15:46.894 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (46013 ms) 2024-03-24T12:15:46.894 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-03-24T12:15:46.894 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (10032 ms) 2024-03-24T12:15:46.894 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-03-24T12:15:46.894 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (7991 ms) 2024-03-24T12:15:46.894 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-03-24T12:15:46.894 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8032 ms) 2024-03-24T12:15:46.895 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-03-24T12:15:46.895 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8312 ms) 2024-03-24T12:15:46.895 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-03-24T12:15:46.895 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (9802 ms) 2024-03-24T12:15:46.895 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-03-24T12:15:46.895 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8866 ms) 2024-03-24T12:15:46.895 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-03-24T12:15:46.895 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10325 ms) 2024-03-24T12:15:46.895 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-03-24T12:15:46.895 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (11690 ms) 2024-03-24T12:15:46.895 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-03-24T12:15:46.895 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (8245 ms) 2024-03-24T12:15:46.895 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-03-24T12:15:46.895 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (7780 ms) 2024-03-24T12:15:46.895 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-03-24T12:15:46.896 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (13050 ms) 2024-03-24T12:15:46.896 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-03-24T12:15:46.896 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7918 ms) 2024-03-24T12:15:46.896 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-03-24T12:15:46.896 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8029 ms) 2024-03-24T12:15:47.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:46 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:15:47.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:46 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-39"}]': finished 2024-03-24T12:15:47.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:46 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi006-81794-55"}]': finished 2024-03-24T12:15:47.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:46 smithi090 ceph-mon[33646]: osdmap e291: 8 total, 8 up, 8 in 2024-03-24T12:15:47.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:46 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi006-81794-55"}]: dispatch 2024-03-24T12:15:47.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:46 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:15:47.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:46 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-39"}]': finished 2024-03-24T12:15:47.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:46 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi006-81794-55"}]': finished 2024-03-24T12:15:47.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:46 smithi006 ceph-mon[30108]: osdmap e291: 8 total, 8 up, 8 in 2024-03-24T12:15:47.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:46 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi006-81794-55"}]: dispatch 2024-03-24T12:15:47.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:46 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:15:47.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:46 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-39"}]': finished 2024-03-24T12:15:47.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:46 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi006-81794-55"}]': finished 2024-03-24T12:15:47.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:46 smithi006 ceph-mon[41734]: osdmap e291: 8 total, 8 up, 8 in 2024-03-24T12:15:47.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:46 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi006-81794-55"}]: dispatch 2024-03-24T12:15:48.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi006-81794-55"}]': finished 2024-03-24T12:15:48.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:47 smithi090 ceph-mon[33646]: osdmap e292: 8 total, 8 up, 8 in 2024-03-24T12:15:48.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3543265905' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:48.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:47 smithi090 ceph-mon[33646]: pgmap v396: 260 pgs: 260 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 976 KiB/s wr, 0 op/s 2024-03-24T12:15:48.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:47 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:48.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3543265905' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:48.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:47 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:48.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3543265905' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi006-81794-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:48.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:47 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi006-81794-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:48.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:47 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:15:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi006-81794-55"}]': finished 2024-03-24T12:15:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:47 smithi006 ceph-mon[30108]: osdmap e292: 8 total, 8 up, 8 in 2024-03-24T12:15:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3543265905' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:47 smithi006 ceph-mon[30108]: pgmap v396: 260 pgs: 260 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 976 KiB/s wr, 0 op/s 2024-03-24T12:15:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:47 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3543265905' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:47 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3543265905' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi006-81794-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:47 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi006-81794-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:48.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:47 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:15:48.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2748008509' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi006-81794-55"}]': finished 2024-03-24T12:15:48.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:47 smithi006 ceph-mon[41734]: osdmap e292: 8 total, 8 up, 8 in 2024-03-24T12:15:48.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3543265905' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:48.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:47 smithi006 ceph-mon[41734]: pgmap v396: 260 pgs: 260 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 976 KiB/s wr, 0 op/s 2024-03-24T12:15:48.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:47 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:48.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3543265905' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:48.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:47 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:48.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3543265905' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi006-81794-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:48.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:47 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi006-81794-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:48.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:47 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:15:48.493 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetrted 0x55ac4adae130 2024-03-24T12:15:48.494 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaea20 2024-03-24T12:15:48.494 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adae580 2024-03-24T12:15:48.494 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adae6b0 2024-03-24T12:15:48.494 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adae800 2024-03-24T12:15:48.494 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adae910 2024-03-24T12:15:48.494 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaeff0 2024-03-24T12:15:48.494 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaeb50 2024-03-24T12:15:48.494 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaec80 2024-03-24T12:15:48.495 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaedb0 2024-03-24T12:15:48.495 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaeee0 2024-03-24T12:15:48.495 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaf5c0 2024-03-24T12:15:48.495 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaf120 2024-03-24T12:15:48.495 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaf250 2024-03-24T12:15:48.495 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaf380 2024-03-24T12:15:48.495 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaf4b0 2024-03-24T12:15:48.495 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adafb90 2024-03-24T12:15:48.495 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaf6f0 2024-03-24T12:15:48.495 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaf840 2024-03-24T12:15:48.495 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaf950 2024-03-24T12:15:48.495 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adafa80 2024-03-24T12:15:48.495 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb0160 2024-03-24T12:15:48.496 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adafcc0 2024-03-24T12:15:48.496 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adafdf0 2024-03-24T12:15:48.496 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaff00 2024-03-24T12:15:48.496 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb0050 2024-03-24T12:15:48.496 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb0730 2024-03-24T12:15:48.496 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb0290 2024-03-24T12:15:48.496 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb03c0 2024-03-24T12:15:48.497 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb04d0 2024-03-24T12:15:48.497 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb0600 2024-03-24T12:15:48.497 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb0d00 2024-03-24T12:15:48.497 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb0860 2024-03-24T12:15:48.497 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb0970 2024-03-24T12:15:48.497 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb0ac0 2024-03-24T12:15:48.497 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb0bf0 2024-03-24T12:15:48.497 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb1330 2024-03-24T12:15:48.497 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb0e90 2024-03-24T12:15:48.497 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb0fc0 2024-03-24T12:15:48.497 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb10d0 2024-03-24T12:15:48.497 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb1220 2024-03-24T12:15:48.497 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb1900 2024-03-24T12:15:48.497 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb1460 2024-03-24T12:15:48.497 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb1590 2024-03-24T12:15:48.498 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb16a0 2024-03-24T12:15:48.498 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb17d0 2024-03-24T12:15:48.498 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb1ed0 2024-03-24T12:15:48.498 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb1a30 2024-03-24T12:15:48.498 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb1b60 2024-03-24T12:15:48.498 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb1c70 2024-03-24T12:15:48.498 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb1dc0 2024-03-24T12:15:48.498 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb24a0 2024-03-24T12:15:48.498 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb2000 2024-03-24T12:15:48.498 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb2130 2024-03-24T12:15:48.498 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb2240 2024-03-24T12:15:48.498 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb2390 2024-03-24T12:15:48.498 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb2a70 2024-03-24T12:15:48.498 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb25d0 2024-03-24T12:15:48.499 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb2700 2024-03-24T12:15:48.499 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb2810 2024-03-24T12:15:48.499 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb2960 2024-03-24T12:15:48.499 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb3080 2024-03-24T12:15:48.499 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb2be0 2024-03-24T12:15:48.499 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb2d10 2024-03-24T12:15:48.499 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb2e20 2024-03-24T12:15:48.499 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb2f70 2024-03-24T12:15:48.499 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb3670 2024-03-24T12:15:48.499 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb31d0 2024-03-24T12:15:48.499 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb3300 2024-03-24T12:15:48.499 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb3430 2024-03-24T12:15:48.499 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb3560 2024-03-24T12:15:48.499 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb3c60 2024-03-24T12:15:48.499 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb37c0 2024-03-24T12:15:48.499 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb38f0 2024-03-24T12:15:48.500 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb3a00 2024-03-24T12:15:48.500 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb3b50 2024-03-24T12:15:48.500 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb4250 2024-03-24T12:15:48.500 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb3db0 2024-03-24T12:15:48.500 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb3ee0 2024-03-24T12:15:48.500 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb3ff0 2024-03-24T12:15:48.500 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb4140 2024-03-24T12:15:48.500 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb4840 2024-03-24T12:15:49.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:49 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi006-81794-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:49.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:49 smithi090 ceph-mon[33646]: osdmap e293: 8 total, 8 up, 8 in 2024-03-24T12:15:49.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:49 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3543265905' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi006-81794-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:49.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:49 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi006-81794-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:49.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:49 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:49.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:49 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:49.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:49 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi006-81794-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:49.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:49 smithi006 ceph-mon[30108]: osdmap e293: 8 total, 8 up, 8 in 2024-03-24T12:15:49.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:49 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3543265905' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi006-81794-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:49.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:49 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi006-81794-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:49.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:49 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:49.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:49 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:49.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:49 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi006-81794-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:49.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:49 smithi006 ceph-mon[41734]: osdmap e293: 8 total, 8 up, 8 in 2024-03-24T12:15:49.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:49 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3543265905' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi006-81794-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:49.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:49 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi006-81794-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:49.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:49 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:49.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:49 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:50.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:50 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:50.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:50 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:50.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:50 smithi090 ceph-mon[33646]: pgmap v399: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-24T12:15:50.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:50 smithi090 ceph-mon[33646]: osdmap e294: 8 total, 8 up, 8 in 2024-03-24T12:15:50.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:50 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:50.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:50 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi006-81794-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi006-81794-56"}]': finished 2024-03-24T12:15:50.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:50 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-41", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:15:50.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:50 smithi090 ceph-mon[33646]: osdmap e295: 8 total, 8 up, 8 in 2024-03-24T12:15:50.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:50 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:15:50.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:50 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:15:50.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:50 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:50.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:50 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:50.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:50 smithi006 ceph-mon[30108]: pgmap v399: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-24T12:15:50.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:50 smithi006 ceph-mon[30108]: osdmap e294: 8 total, 8 up, 8 in 2024-03-24T12:15:50.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:50 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:50.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:50 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi006-81794-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi006-81794-56"}]': finished 2024-03-24T12:15:50.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:50 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-41", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:15:50.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:50 smithi006 ceph-mon[30108]: osdmap e295: 8 total, 8 up, 8 in 2024-03-24T12:15:50.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:50 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:15:50.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:50 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:15:50.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:50 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:50.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:50 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:50.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:50 smithi006 ceph-mon[41734]: pgmap v399: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-24T12:15:50.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:50 smithi006 ceph-mon[41734]: osdmap e294: 8 total, 8 up, 8 in 2024-03-24T12:15:50.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:50 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:50.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:50 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi006-81794-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi006-81794-56"}]': finished 2024-03-24T12:15:50.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:50 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-41", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:15:50.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:50 smithi006 ceph-mon[41734]: osdmap e295: 8 total, 8 up, 8 in 2024-03-24T12:15:50.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:50 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:15:50.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:50 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:15:51.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:51 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:51.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:51 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:15:51.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:51 smithi090 ceph-mon[33646]: osdmap e296: 8 total, 8 up, 8 in 2024-03-24T12:15:51.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:51 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:15:51.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:51 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:15:51.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:51 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:51.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:51 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:15:51.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:51 smithi006 ceph-mon[30108]: osdmap e296: 8 total, 8 up, 8 in 2024-03-24T12:15:51.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:15:51.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:51 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:15:51.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:51 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:51.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:51 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:15:51.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:51 smithi006 ceph-mon[41734]: osdmap e296: 8 total, 8 up, 8 in 2024-03-24T12:15:51.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:51 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:15:51.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:51 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:15:52.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:15:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:15:52.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:52 smithi090 ceph-mon[33646]: pgmap v402: 300 pgs: 40 unknown, 260 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:52.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:52 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:15:52.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:52 smithi090 ceph-mon[33646]: osdmap e297: 8 total, 8 up, 8 in 2024-03-24T12:15:52.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:52 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-24T12:15:52.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:52 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-24T12:15:52.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:52 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3543265905' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:52.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:52 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:52.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:52 smithi006 ceph-mon[30108]: pgmap v402: 300 pgs: 40 unknown, 260 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:52.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:52 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:15:52.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:52 smithi006 ceph-mon[30108]: osdmap e297: 8 total, 8 up, 8 in 2024-03-24T12:15:52.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:52 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-24T12:15:52.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:52 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-24T12:15:52.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:52 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3543265905' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:52.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:52 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:52.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:52 smithi006 ceph-mon[41734]: pgmap v402: 300 pgs: 40 unknown, 260 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:52.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:52 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:15:52.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:52 smithi006 ceph-mon[41734]: osdmap e297: 8 total, 8 up, 8 in 2024-03-24T12:15:52.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:52 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-24T12:15:52.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:52 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-24T12:15:52.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:52 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3543265905' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:52.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:52 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:53.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:53 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:15:53.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:53 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-24T12:15:53.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:53 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi006-81794-56"}]': finished 2024-03-24T12:15:53.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:53 smithi090 ceph-mon[33646]: osdmap e298: 8 total, 8 up, 8 in 2024-03-24T12:15:53.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:53 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3543265905' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:53.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:53 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:53.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:53 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:15:53.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:53 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-24T12:15:53.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:53 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi006-81794-56"}]': finished 2024-03-24T12:15:53.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:53 smithi006 ceph-mon[30108]: osdmap e298: 8 total, 8 up, 8 in 2024-03-24T12:15:53.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:53 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3543265905' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:53.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:53 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:53.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:53 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:15:53.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:53 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-24T12:15:53.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:53 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi006-81794-56"}]': finished 2024-03-24T12:15:53.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:53 smithi006 ceph-mon[41734]: osdmap e298: 8 total, 8 up, 8 in 2024-03-24T12:15:53.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:53 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3543265905' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:53.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:53 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi006-81794-56"}]: dispatch 2024-03-24T12:15:54.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:54 smithi090 ceph-mon[33646]: pgmap v405: 292 pgs: 292 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:54.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:54 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi006-81794-56"}]': finished 2024-03-24T12:15:54.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:54 smithi090 ceph-mon[33646]: osdmap e299: 8 total, 8 up, 8 in 2024-03-24T12:15:54.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3964227160' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:15:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:54 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:15:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3964227160' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:15:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:54 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:15:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3964227160' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi006-81794-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:54 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi006-81794-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:54 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-41"}]: dispatch 2024-03-24T12:15:54.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:54 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-41"}]: dispatch 2024-03-24T12:15:54.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[30108]: pgmap v405: 292 pgs: 292 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:54.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi006-81794-56"}]': finished 2024-03-24T12:15:54.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[30108]: osdmap e299: 8 total, 8 up, 8 in 2024-03-24T12:15:54.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3964227160' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:15:54.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:15:54.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3964227160' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:15:54.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:15:54.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3964227160' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi006-81794-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:54.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi006-81794-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:54.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:54.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:54.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-41"}]: dispatch 2024-03-24T12:15:54.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-41"}]: dispatch 2024-03-24T12:15:54.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[41734]: pgmap v405: 292 pgs: 292 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:54.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi006-81794-56"}]': finished 2024-03-24T12:15:54.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[41734]: osdmap e299: 8 total, 8 up, 8 in 2024-03-24T12:15:54.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3964227160' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:15:54.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:15:54.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3964227160' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:15:54.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:15:54.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3964227160' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi006-81794-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:54.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi006-81794-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:15:54.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:54.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:15:54.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-41"}]: dispatch 2024-03-24T12:15:54.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:54 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-41"}]: dispatch 2024-03-24T12:15:55.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:15:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:15:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:15:56.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:55 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi006-81794-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:56.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:55 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-41"}]': finished 2024-03-24T12:15:56.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:55 smithi006 ceph-mon[30108]: osdmap e300: 8 total, 8 up, 8 in 2024-03-24T12:15:56.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:55 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3964227160' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi006-81794-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:15:56.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:55 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi006-81794-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:15:56.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:55 smithi006 ceph-mon[30108]: pgmap v408: 292 pgs: 292 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:56.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:55 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:15:56.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:55 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:15:56.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:55 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:15:56.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:55 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi006-81794-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:56.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:55 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-41"}]': finished 2024-03-24T12:15:56.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:55 smithi006 ceph-mon[41734]: osdmap e300: 8 total, 8 up, 8 in 2024-03-24T12:15:56.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:55 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3964227160' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi006-81794-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:15:56.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:55 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi006-81794-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:15:56.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:55 smithi006 ceph-mon[41734]: pgmap v408: 292 pgs: 292 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:56.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:55 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:15:56.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:55 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:15:56.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:55 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:15:56.230 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:55 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi006-81794-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:15:56.230 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:55 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-41"}]': finished 2024-03-24T12:15:56.230 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:55 smithi090 ceph-mon[33646]: osdmap e300: 8 total, 8 up, 8 in 2024-03-24T12:15:56.230 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:55 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3964227160' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi006-81794-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:15:56.231 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:55 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi006-81794-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:15:56.231 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:55 smithi090 ceph-mon[33646]: pgmap v408: 292 pgs: 292 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:15:56.231 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:55 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:15:56.231 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:55 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:15:56.231 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:55 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:15:57.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:56 smithi006 ceph-mon[30108]: osdmap e301: 8 total, 8 up, 8 in 2024-03-24T12:15:57.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:56 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:15:57.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:56 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:15:57.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:56 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:15:57.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:56 smithi006 ceph-mon[41734]: osdmap e301: 8 total, 8 up, 8 in 2024-03-24T12:15:57.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:56 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:15:57.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:56 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:15:57.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:56 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:15:57.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:56 smithi090 ceph-mon[33646]: osdmap e301: 8 total, 8 up, 8 in 2024-03-24T12:15:57.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:56 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:15:57.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:56 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:15:57.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:56 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:15:58.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:57 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi006-81794-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi006-81794-57"}]': finished 2024-03-24T12:15:58.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:57 smithi006 ceph-mon[30108]: osdmap e302: 8 total, 8 up, 8 in 2024-03-24T12:15:58.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:57 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:58.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:57 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:58.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:57 smithi006 ceph-mon[30108]: pgmap v411: 300 pgs: 24 creating+peering, 16 unknown, 260 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:15:58.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:57 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi006-81794-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi006-81794-57"}]': finished 2024-03-24T12:15:58.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:57 smithi006 ceph-mon[41734]: osdmap e302: 8 total, 8 up, 8 in 2024-03-24T12:15:58.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:57 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:58.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:57 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:58.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:57 smithi006 ceph-mon[41734]: pgmap v411: 300 pgs: 24 creating+peering, 16 unknown, 260 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:15:58.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:57 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi006-81794-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi006-81794-57"}]': finished 2024-03-24T12:15:58.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:57 smithi090 ceph-mon[33646]: osdmap e302: 8 total, 8 up, 8 in 2024-03-24T12:15:58.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:57 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:58.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:57 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:15:58.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:57 smithi090 ceph-mon[33646]: pgmap v411: 300 pgs: 24 creating+peering, 16 unknown, 260 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:15:59.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:58 smithi006 ceph-mon[30108]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:59.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:58 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:59.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:58 smithi006 ceph-mon[30108]: osdmap e303: 8 total, 8 up, 8 in 2024-03-24T12:15:59.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:58 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi006-82061-6","var": "pg_num","format": "json"}]: dispatch 2024-03-24T12:15:59.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:58 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:59.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:58 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:59.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:58 smithi006 ceph-mon[41734]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:59.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:58 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:59.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:58 smithi006 ceph-mon[41734]: osdmap e303: 8 total, 8 up, 8 in 2024-03-24T12:15:59.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:58 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi006-82061-6","var": "pg_num","format": "json"}]: dispatch 2024-03-24T12:15:59.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:58 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:59.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:58 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:59.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:58 smithi090 ceph-mon[33646]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:15:59.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:58 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:15:59.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:58 smithi090 ceph-mon[33646]: osdmap e303: 8 total, 8 up, 8 in 2024-03-24T12:15:59.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:58 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi006-82061-6","var": "pg_num","format": "json"}]: dispatch 2024-03-24T12:15:59.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:58 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:15:59.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:58 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:16:00.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:59 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-43", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:16:00.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:59 smithi090 ceph-mon[33646]: osdmap e304: 8 total, 8 up, 8 in 2024-03-24T12:16:00.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:59 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-24T12:16:00.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:59 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-24T12:16:00.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:59 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3964227160' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:16:00.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:59 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:16:00.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:59 smithi090 ceph-mon[33646]: pgmap v414: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:00.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:15:59 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_count","val": "8"}]': finished 2024-03-24T12:16:00.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:59 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-43", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:16:00.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:59 smithi006 ceph-mon[30108]: osdmap e304: 8 total, 8 up, 8 in 2024-03-24T12:16:00.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:59 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-24T12:16:00.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:59 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-24T12:16:00.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:59 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3964227160' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:16:00.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:59 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:16:00.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:59 smithi006 ceph-mon[30108]: pgmap v414: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:00.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:15:59 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_count","val": "8"}]': finished 2024-03-24T12:16:00.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:59 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-43", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:16:00.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:59 smithi006 ceph-mon[41734]: osdmap e304: 8 total, 8 up, 8 in 2024-03-24T12:16:00.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:59 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-24T12:16:00.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:59 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-24T12:16:00.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:59 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3964227160' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:16:00.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:59 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:16:00.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:59 smithi006 ceph-mon[41734]: pgmap v414: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:00.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:15:59 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_count","val": "8"}]': finished 2024-03-24T12:16:01.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:00 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi006-81794-57"}]': finished 2024-03-24T12:16:01.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:00 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:16:01.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:00 smithi090 ceph-mon[33646]: osdmap e305: 8 total, 8 up, 8 in 2024-03-24T12:16:01.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:00 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3964227160' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:16:01.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:00 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:16:01.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:00 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:16:01.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:00 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:16:01.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:00 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi006-81794-57"}]': finished 2024-03-24T12:16:01.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:00 smithi090 ceph-mon[33646]: osdmap e306: 8 total, 8 up, 8 in 2024-03-24T12:16:01.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:00 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-24T12:16:01.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:00 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-24T12:16:01.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:00 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi006-81794-57"}]': finished 2024-03-24T12:16:01.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:00 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:16:01.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:00 smithi006 ceph-mon[30108]: osdmap e305: 8 total, 8 up, 8 in 2024-03-24T12:16:01.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:00 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3964227160' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:16:01.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:00 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:16:01.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:00 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:16:01.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:00 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:16:01.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:00 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi006-81794-57"}]': finished 2024-03-24T12:16:01.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:00 smithi006 ceph-mon[30108]: osdmap e306: 8 total, 8 up, 8 in 2024-03-24T12:16:01.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:00 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-24T12:16:01.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:00 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-24T12:16:01.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:00 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi006-81794-57"}]': finished 2024-03-24T12:16:01.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:00 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:16:01.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:00 smithi006 ceph-mon[41734]: osdmap e305: 8 total, 8 up, 8 in 2024-03-24T12:16:01.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:00 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3964227160' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:16:01.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:00 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:16:01.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:00 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi006-81794-57"}]: dispatch 2024-03-24T12:16:01.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:00 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:16:01.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:00 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi006-81794-57"}]': finished 2024-03-24T12:16:01.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:00 smithi006 ceph-mon[41734]: osdmap e306: 8 total, 8 up, 8 in 2024-03-24T12:16:01.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:00 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-24T12:16:01.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:00 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-24T12:16:02.200 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc:Read 2024-03-24T12:16:02.200 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: hmm, no HitSet yet 2024-03-24T12:16:02.201 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: ok, hit_set contains 274:602f83fe:::foo:head 2024-03-24T12:16:02.201 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9304 ms) 2024-03-24T12:16:02.201 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-03-24T12:16:02.201 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg_num = 32 2024-03-24T12:16:02.201 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 0 ls 1711282562,0 2024-03-24T12:16:02.201 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 1 ls 1711282562,0 2024-03-24T12:16:02.201 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 2 ls 1711282562,0 2024-03-24T12:16:02.201 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 3 ls 1711282562,0 2024-03-24T12:16:02.201 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 4 ls 1711282562,0 2024-03-24T12:16:02.201 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 5 ls 1711282562,0 2024-03-24T12:16:02.201 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 6 ls 1711282562,0 2024-03-24T12:16:02.201 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 7 ls 1711282562,0 2024-03-24T12:16:02.201 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 8 ls 1711282562,0 2024-03-24T12:16:02.201 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 9 ls 1711282562,0 2024-03-24T12:16:02.202 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 10 ls 1711282562,0 2024-03-24T12:16:02.202 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 11 ls 1711282562,0 2024-03-24T12:16:02.202 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 12 ls 1711282562,0 2024-03-24T12:16:02.202 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 13 ls 1711282562,0 2024-03-24T12:16:02.202 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 14 ls 1711282562,0 2024-03-24T12:16:02.202 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 15 ls 1711282562,0 2024-03-24T12:16:02.202 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 16 ls 1711282562,0 2024-03-24T12:16:02.202 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 17 ls 1711282562,0 2024-03-24T12:16:02.202 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 18 ls 1711282562,0 2024-03-24T12:16:02.202 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 19 ls 1711282562,0 2024-03-24T12:16:02.202 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 20 ls 1711282562,0 2024-03-24T12:16:02.202 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 21 ls 1711282562,0 2024-03-24T12:16:02.203 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 22 ls 1711282562,0 2024-03-24T12:16:02.203 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 23 ls 1711282562,0 2024-03-24T12:16:02.203 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 24 ls 1711282562,0 2024-03-24T12:16:02.203 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 25 ls 1711282562,0 2024-03-24T12:16:02.203 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 26 ls 1711282562,0 2024-03-24T12:16:02.203 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 27 ls 1711282562,0 2024-03-24T12:16:02.203 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 28 ls 1711282562,0 2024-03-24T12:16:02.203 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 29 ls 1711282562,0 2024-03-24T12:16:02.203 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 30 ls 1711282562,0 2024-03-24T12:16:02.203 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg 31 ls 1711282562,0 2024-03-24T12:16:02.203 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: pg_num = 32 2024-03-24T12:16:02.203 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:6cac518f:::0:head 2024-03-24T12:16:02.203 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:02547ec2:::1:head 2024-03-24T12:16:02.205 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:f905c69b:::2:head 2024-03-24T12:16:02.205 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:cfc208b3:::3:head 2024-03-24T12:16:02.205 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:d83876eb:::4:head 2024-03-24T12:16:02.205 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:b29083e3:::5:head 2024-03-24T12:16:02.205 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:c4fdafeb:::6:head 2024-03-24T12:16:02.205 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:5c6b0b28:::7:head 2024-03-24T12:16:02.205 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:bd63b0f1:::8:head 2024-03-24T12:16:02.205 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:e960b815:::9:head 2024-03-24T12:16:02.205 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:52ea6a34:::10:head 2024-03-24T12:16:02.206 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:89d3ae78:::11:head 2024-03-24T12:16:02.206 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:de5d7c5f:::12:head 2024-03-24T12:16:02.206 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:566253c9:::13:head 2024-03-24T12:16:02.206 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:62a1935d:::14:head 2024-03-24T12:16:02.206 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:863748b0:::15:head 2024-03-24T12:16:02.206 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:3958e169:::16:head 2024-03-24T12:16:02.206 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:4d4dabf9:::17:head 2024-03-24T12:16:02.206 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:8391935d:::18:head 2024-03-24T12:16:02.206 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:28883081:::19:head 2024-03-24T12:16:02.206 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:69259c59:::20:head 2024-03-24T12:16:02.206 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:4bdb80b7:::21:head 2024-03-24T12:16:02.206 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:a11c5d71:::22:head 2024-03-24T12:16:02.207 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:271af37b:::23:head 2024-03-24T12:16:02.207 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:95b121be:::24:head 2024-03-24T12:16:02.207 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:58d1031b:::25:head 2024-03-24T12:16:02.207 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:0a050783:::26:head 2024-03-24T12:16:02.207 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:c709704c:::27:head 2024-03-24T12:16:02.207 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:cbe56eaf:::28:head 2024-03-24T12:16:02.207 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:86b4b162:::29:head 2024-03-24T12:16:02.207 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:70d89383:::30:head 2024-03-24T12:16:02.207 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:dd450c7c:::31:head 2024-03-24T12:16:02.207 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:6d5729b1:::32:head 2024-03-24T12:16:02.207 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:c388f3fb:::33:head 2024-03-24T12:16:02.207 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:56cfea31:::34:head 2024-03-24T12:16:02.207 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:9dbc1bf7:::35:head 2024-03-24T12:16:02.207 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:40b74ccd:::36:head 2024-03-24T12:16:02.208 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:4d5aaf42:::37:head 2024-03-24T12:16:02.208 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:920f362c:::38:head 2024-03-24T12:16:02.208 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:6cc53222:::39:head 2024-03-24T12:16:02.208 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:9cad833f:::40:head 2024-03-24T12:16:02.208 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:1ea84d41:::41:head 2024-03-24T12:16:02.208 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:c4480ef6:::42:head 2024-03-24T12:16:02.208 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:a694361e:::43:head 2024-03-24T12:16:02.208 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:d1bd33e9:::44:head 2024-03-24T12:16:02.208 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:ddc2cd5d:::45:head 2024-03-24T12:16:02.208 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:2b782207:::46:head 2024-03-24T12:16:02.208 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:7b187fca:::47:head 2024-03-24T12:16:02.208 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:90ecdf6f:::48:head 2024-03-24T12:16:02.208 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:a5ed95fe:::49:head 2024-03-24T12:16:02.208 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:ea0eaa55:::50:head 2024-03-24T12:16:02.209 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:f33ef17b:::51:head 2024-03-24T12:16:02.209 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:a0d1b2f6:::52:head 2024-03-24T12:16:02.209 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:60c5229e:::53:head 2024-03-24T12:16:02.209 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:edcbc575:::54:head 2024-03-24T12:16:02.209 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:102cf253:::55:head 2024-03-24T12:16:02.209 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:efb7fb0b:::56:head 2024-03-24T12:16:02.209 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:50d0a326:::57:head 2024-03-24T12:16:02.209 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:d4dc5daf:::58:head 2024-03-24T12:16:02.209 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:3a130462:::59:head 2024-03-24T12:16:02.209 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:ec87ed71:::60:head 2024-03-24T12:16:02.209 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:d5bc9454:::61:head 2024-03-24T12:16:02.209 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:3ddfe313:::62:head 2024-03-24T12:16:02.209 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:7c2816b9:::63:head 2024-03-24T12:16:02.209 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:47e00e4d:::64:head 2024-03-24T12:16:02.209 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:c6410c18:::65:head 2024-03-24T12:16:02.210 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:b48ed237:::66:head 2024-03-24T12:16:02.210 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:cd63ad31:::67:head 2024-03-24T12:16:02.210 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:b179e92b:::68:head 2024-03-24T12:16:02.210 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:0d9f741a:::69:head 2024-03-24T12:16:02.210 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:6d3352ae:::70:head 2024-03-24T12:16:02.210 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:c6d5c19e:::71:head 2024-03-24T12:16:02.210 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:bc4729c3:::72:head 2024-03-24T12:16:02.210 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:77e930b9:::73:head 2024-03-24T12:16:02.210 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:0abeecfd:::74:head 2024-03-24T12:16:02.210 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:b7c37e15:::75:head 2024-03-24T12:16:02.210 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:b6378398:::76:head 2024-03-24T12:16:02.210 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:02bd68de:::77:head 2024-03-24T12:16:02.210 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:cc795d2d:::78:head 2024-03-24T12:16:02.210 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:630d4fea:::79:head 2024-03-24T12:16:02.211 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:e0d29ef5:::80:head 2024-03-24T12:16:02.211 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:fd6f13d2:::81:head 2024-03-24T12:16:02.211 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:606461d5:::82:head 2024-03-24T12:16:02.211 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:eadbdc43:::83:head 2024-03-24T12:16:02.211 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:8761d0bb:::84:head 2024-03-24T12:16:02.211 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:9ef0186f:::85:head 2024-03-24T12:16:02.211 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:e0d41294:::86:head 2024-03-24T12:16:02.211 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:961de695:::87:head 2024-03-24T12:16:02.211 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:1423148f:::88:head 2024-03-24T12:16:02.211 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:633a8fa2:::89:head 2024-03-24T12:16:02.211 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:a8653809:::90:head 2024-03-24T12:16:02.211 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:3dac8b33:::91:head 2024-03-24T12:16:02.211 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:35aad435:::92:head 2024-03-24T12:16:02.211 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:f6dcc343:::93:head 2024-03-24T12:16:02.211 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:dbbdad87:::94:head 2024-03-24T12:16:02.212 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:1cb48ce0:::95:head 2024-03-24T12:16:02.212 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:03cd461c:::96:head 2024-03-24T12:16:02.212 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:17a4ea99:::97:head 2024-03-24T12:16:02.212 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:9993c9a7:::98:head 2024-03-24T12:16:02.212 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:6394211c:::99:head 2024-03-24T12:16:02.212 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:94c7ae57:::100:head 2024-03-24T12:16:02.212 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:6fdee5bb:::101:head 2024-03-24T12:16:02.212 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:9a477fd1:::102:head 2024-03-24T12:16:02.212 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:eb850916:::103:head 2024-03-24T12:16:02.212 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:affc56b9:::104:head 2024-03-24T12:16:02.212 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:b42dc814:::105:head 2024-03-24T12:16:02.212 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:f319f8f0:::106:head 2024-03-24T12:16:02.212 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:9a40b9de:::107:head 2024-03-24T12:16:02.212 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:8b524f28:::108:head 2024-03-24T12:16:02.213 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:e3de589f:::109:head 2024-03-24T12:16:02.213 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:90f90a5b:::110:head 2024-03-24T12:16:02.213 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:a7b4f1d7:::111:head 2024-03-24T12:16:02.213 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:af51766e:::112:head 2024-03-24T12:16:02.213 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:b6f90bd1:::113:head 2024-03-24T12:16:02.213 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:e0261208:::114:head 2024-03-24T12:16:02.213 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:c9569ef7:::115:head 2024-03-24T12:16:02.213 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:61bebe50:::116:head 2024-03-24T12:16:02.213 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:fe93412b:::117:head 2024-03-24T12:16:02.213 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:d3d38bee:::118:head 2024-03-24T12:16:02.213 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:3100ba0c:::119:head 2024-03-24T12:16:02.213 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:d0560ada:::120:head 2024-03-24T12:16:02.213 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:f0ea8b35:::121:head 2024-03-24T12:16:02.213 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:766f231a:::122:head 2024-03-24T12:16:02.214 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:a07a2582:::123:head 2024-03-24T12:16:02.214 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:bd7c6b3a:::124:head 2024-03-24T12:16:02.214 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:fb2ddaff:::125:head 2024-03-24T12:16:02.214 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:4408e1fe:::126:head 2024-03-24T12:16:02.214 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:ee1df7a7:::127:head 2024-03-24T12:16:02.214 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:c3002909:::128:head 2024-03-24T12:16:02.215 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:4f48ffa9:::129:head 2024-03-24T12:16:02.215 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:edf38733:::130:head 2024-03-24T12:16:02.215 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:c08425c0:::131:head 2024-03-24T12:16:02.215 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:5f902d98:::132:head 2024-03-24T12:16:02.215 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:41ea2c93:::133:head 2024-03-24T12:16:02.215 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:813cee13:::134:head 2024-03-24T12:16:02.215 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:0131818d:::135:head 2024-03-24T12:16:02.215 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:26ba5a85:::136:head 2024-03-24T12:16:02.215 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:381b8a5a:::137:head 2024-03-24T12:16:02.215 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:28797e47:::138:head 2024-03-24T12:16:02.215 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:bfca7f22:::139:head 2024-03-24T12:16:02.215 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:36807075:::140:head 2024-03-24T12:16:02.215 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:80b03975:::141:head 2024-03-24T12:16:02.216 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:5c15709b:::142:head 2024-03-24T12:16:02.216 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:f39ea15e:::143:head 2024-03-24T12:16:02.216 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:ea992956:::144:head 2024-03-24T12:16:02.216 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:48887b1c:::145:head 2024-03-24T12:16:02.216 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:9f24a9dd:::146:head 2024-03-24T12:16:02.216 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:987f100b:::147:head 2024-03-24T12:16:02.216 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:d2dd3581:::148:head 2024-03-24T12:16:02.216 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:7fed1808:::149:head 2024-03-24T12:16:02.216 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:c80b70e9:::150:head 2024-03-24T12:16:02.216 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:85ed90f9:::151:head 2024-03-24T12:16:02.216 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:36428b24:::152:head 2024-03-24T12:16:02.217 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:d044c34a:::153:head 2024-03-24T12:16:02.217 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:7c18bf58:::154:head 2024-03-24T12:16:02.217 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:d1c21232:::155:head 2024-03-24T12:16:02.217 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:a7a3c575:::156:head 2024-03-24T12:16:02.217 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:87da0633:::157:head 2024-03-24T12:16:02.217 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:d5ac3822:::158:head 2024-03-24T12:16:02.217 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:3f20522d:::159:head 2024-03-24T12:16:02.217 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:6ca26563:::160:head 2024-03-24T12:16:02.217 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:532ce135:::161:head 2024-03-24T12:16:02.217 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:c78863e6:::162:head 2024-03-24T12:16:02.334 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi006-81794-58"}]: dispatch 2024-03-24T12:16:02.334 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi006-81794-58"}]: dispatch 2024-03-24T12:16:02.334 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi006-81794-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:02.335 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:01 smithi090 ceph-mon[33646]: pgmap v417: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:02.335 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-03-24T12:16:02.335 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi006-81794-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:02.335 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:01 smithi090 ceph-mon[33646]: osdmap e307: 8 total, 8 up, 8 in 2024-03-24T12:16:02.335 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi006-81794-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi006-81794-58"}]: dispatch 2024-03-24T12:16:02.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi006-81794-58"}]: dispatch 2024-03-24T12:16:02.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi006-81794-58"}]: dispatch 2024-03-24T12:16:02.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi006-81794-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:02.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:01 smithi006 ceph-mon[30108]: pgmap v417: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:02.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-03-24T12:16:02.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi006-81794-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:02.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:01 smithi006 ceph-mon[30108]: osdmap e307: 8 total, 8 up, 8 in 2024-03-24T12:16:02.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi006-81794-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi006-81794-58"}]: dispatch 2024-03-24T12:16:02.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi006-81794-58"}]: dispatch 2024-03-24T12:16:02.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi006-81794-58"}]: dispatch 2024-03-24T12:16:02.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi006-81794-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:02.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:01 smithi006 ceph-mon[41734]: pgmap v417: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:02.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-03-24T12:16:02.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi006-81794-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:02.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:01 smithi006 ceph-mon[41734]: osdmap e307: 8 total, 8 up, 8 in 2024-03-24T12:16:02.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi006-81794-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi006-81794-58"}]: dispatch 2024-03-24T12:16:02.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:16:02 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:16:03.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi006-82061-43","var": "pg_num","format": "json"}]: dispatch 2024-03-24T12:16:03.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:03.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:03 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:03.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-43"}]: dispatch 2024-03-24T12:16:03.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:03 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-43"}]: dispatch 2024-03-24T12:16:03.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:03 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:16:03.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:03 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:16:03.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:03 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-43"}]': finished 2024-03-24T12:16:03.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi006-82061-43","var": "pg_num","format": "json"}]: dispatch 2024-03-24T12:16:03.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:03.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:03 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:03.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-43"}]: dispatch 2024-03-24T12:16:03.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:03 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-43"}]: dispatch 2024-03-24T12:16:03.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:03 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:16:03.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:03 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:16:03.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:03 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-43"}]': finished 2024-03-24T12:16:03.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi006-82061-43","var": "pg_num","format": "json"}]: dispatch 2024-03-24T12:16:03.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:03.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:03 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:03.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-43"}]: dispatch 2024-03-24T12:16:03.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:03 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-43"}]: dispatch 2024-03-24T12:16:03.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:03 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:16:03.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:03 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:16:03.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:03 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-43"}]': finished 2024-03-24T12:16:04.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:04 smithi090 ceph-mon[33646]: pgmap v420: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-03-24T12:16:04.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:04 smithi090 ceph-mon[33646]: osdmap e308: 8 total, 8 up, 8 in 2024-03-24T12:16:04.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:04 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi006-81794-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi006-81794-58"}]': finished 2024-03-24T12:16:04.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:04 smithi090 ceph-mon[33646]: osdmap e309: 8 total, 8 up, 8 in 2024-03-24T12:16:04.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:04 smithi006 ceph-mon[30108]: pgmap v420: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-03-24T12:16:04.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:04 smithi006 ceph-mon[30108]: osdmap e308: 8 total, 8 up, 8 in 2024-03-24T12:16:04.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:04 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi006-81794-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi006-81794-58"}]': finished 2024-03-24T12:16:04.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:04 smithi006 ceph-mon[30108]: osdmap e309: 8 total, 8 up, 8 in 2024-03-24T12:16:04.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:04 smithi006 ceph-mon[41734]: pgmap v420: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-03-24T12:16:04.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:04 smithi006 ceph-mon[41734]: osdmap e308: 8 total, 8 up, 8 in 2024-03-24T12:16:04.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:04 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi006-81794-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi006-81794-58"}]': finished 2024-03-24T12:16:04.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:04 smithi006 ceph-mon[41734]: osdmap e309: 8 total, 8 up, 8 in 2024-03-24T12:16:05.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:16:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:16:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:16:06.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:05 smithi090 ceph-mon[33646]: osdmap e310: 8 total, 8 up, 8 in 2024-03-24T12:16:06.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:05 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:06.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:05 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:06.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:05 smithi090 ceph-mon[33646]: pgmap v423: 300 pgs: 40 unknown, 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:06.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:05 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:06.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:05 smithi006 ceph-mon[30108]: osdmap e310: 8 total, 8 up, 8 in 2024-03-24T12:16:06.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:05 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:06.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:05 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:06.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:05 smithi006 ceph-mon[30108]: pgmap v423: 300 pgs: 40 unknown, 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:06.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:05 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:06.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:05 smithi006 ceph-mon[41734]: osdmap e310: 8 total, 8 up, 8 in 2024-03-24T12:16:06.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:05 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:06.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:05 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:06.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:05 smithi006 ceph-mon[41734]: pgmap v423: 300 pgs: 40 unknown, 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:06.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:05 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:07.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:06 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:16:07.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:06 smithi090 ceph-mon[33646]: osdmap e311: 8 total, 8 up, 8 in 2024-03-24T12:16:07.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:06 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:16:07.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:06 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:16:07.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:06 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi006-81794-58"}]: dispatch 2024-03-24T12:16:07.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:06 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:16:07.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:06 smithi006 ceph-mon[30108]: osdmap e311: 8 total, 8 up, 8 in 2024-03-24T12:16:07.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:06 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:16:07.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:06 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:16:07.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:06 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi006-81794-58"}]: dispatch 2024-03-24T12:16:07.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:06 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:16:07.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:06 smithi006 ceph-mon[41734]: osdmap e311: 8 total, 8 up, 8 in 2024-03-24T12:16:07.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:06 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:16:07.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:06 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:16:07.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:06 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi006-81794-58"}]: dispatch 2024-03-24T12:16:08.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:07 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-45", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:16:08.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:07 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi006-81794-58"}]': finished 2024-03-24T12:16:08.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:07 smithi090 ceph-mon[33646]: osdmap e312: 8 total, 8 up, 8 in 2024-03-24T12:16:08.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:07 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi006-81794-58"}]: dispatch 2024-03-24T12:16:08.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:07 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-24T12:16:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:07 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-24T12:16:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:07 smithi090 ceph-mon[33646]: pgmap v426: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:07 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi006-81794-58"}]': finished 2024-03-24T12:16:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:07 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_count","val": "3"}]': finished 2024-03-24T12:16:08.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:07 smithi090 ceph-mon[33646]: osdmap e313: 8 total, 8 up, 8 in 2024-03-24T12:16:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:07 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-45", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:16:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:07 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi006-81794-58"}]': finished 2024-03-24T12:16:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:07 smithi006 ceph-mon[30108]: osdmap e312: 8 total, 8 up, 8 in 2024-03-24T12:16:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:07 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi006-81794-58"}]: dispatch 2024-03-24T12:16:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:07 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-24T12:16:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:07 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-24T12:16:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:07 smithi006 ceph-mon[30108]: pgmap v426: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:07 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi006-81794-58"}]': finished 2024-03-24T12:16:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:07 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_count","val": "3"}]': finished 2024-03-24T12:16:08.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:07 smithi006 ceph-mon[30108]: osdmap e313: 8 total, 8 up, 8 in 2024-03-24T12:16:08.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:07 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-45", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:16:08.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:07 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi006-81794-58"}]': finished 2024-03-24T12:16:08.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:07 smithi006 ceph-mon[41734]: osdmap e312: 8 total, 8 up, 8 in 2024-03-24T12:16:08.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:07 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi006-81794-58"}]: dispatch 2024-03-24T12:16:08.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:07 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-24T12:16:08.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:07 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-24T12:16:08.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:07 smithi006 ceph-mon[41734]: pgmap v426: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:08.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:07 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3670490377' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi006-81794-58"}]': finished 2024-03-24T12:16:08.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:07 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_count","val": "3"}]': finished 2024-03-24T12:16:08.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:07 smithi006 ceph-mon[41734]: osdmap e313: 8 total, 8 up, 8 in 2024-03-24T12:16:09.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-24T12:16:09.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-24T12:16:09.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1685249395' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:09.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:09.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1685249395' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:09.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:09.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1685249395' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:09.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:09.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_period","val": "3"}]': finished 2024-03-24T12:16:09.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:09.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:08 smithi090 ceph-mon[33646]: osdmap e314: 8 total, 8 up, 8 in 2024-03-24T12:16:09.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:16:09.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:16:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-24T12:16:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-24T12:16:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1685249395' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1685249395' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1685249395' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_period","val": "3"}]': finished 2024-03-24T12:16:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:09.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[30108]: osdmap e314: 8 total, 8 up, 8 in 2024-03-24T12:16:09.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:16:09.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:16:09.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-24T12:16:09.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-24T12:16:09.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1685249395' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:09.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:09.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1685249395' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:09.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:09.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1685249395' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:09.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:09.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_period","val": "3"}]': finished 2024-03-24T12:16:09.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:09.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[41734]: osdmap e314: 8 total, 8 up, 8 in 2024-03-24T12:16:09.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:16:09.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:16:10.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1685249395' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi006-81794-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:10.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:10 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi006-81794-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:10.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:10 smithi090 ceph-mon[33646]: pgmap v429: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:10.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:10 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_type","val": "bloom"}]': finished 2024-03-24T12:16:10.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1685249395' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi006-81794-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:10.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:10 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi006-81794-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:10.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:10 smithi006 ceph-mon[30108]: pgmap v429: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:10.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:10 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_type","val": "bloom"}]': finished 2024-03-24T12:16:10.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1685249395' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi006-81794-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:10.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:10 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi006-81794-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:10.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:10 smithi006 ceph-mon[41734]: pgmap v429: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:10.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:10 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_type","val": "bloom"}]': finished 2024-03-24T12:16:11.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-24T12:16:11.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:11 smithi090 ceph-mon[33646]: osdmap e315: 8 total, 8 up, 8 in 2024-03-24T12:16:11.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-24T12:16:11.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi006-81794-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]': finished 2024-03-24T12:16:11.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-24T12:16:11.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:11 smithi090 ceph-mon[33646]: osdmap e316: 8 total, 8 up, 8 in 2024-03-24T12:16:11.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-24T12:16:11.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:11 smithi006 ceph-mon[30108]: osdmap e315: 8 total, 8 up, 8 in 2024-03-24T12:16:11.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-24T12:16:11.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi006-81794-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]': finished 2024-03-24T12:16:11.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-24T12:16:11.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:11 smithi006 ceph-mon[30108]: osdmap e316: 8 total, 8 up, 8 in 2024-03-24T12:16:11.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-24T12:16:11.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:11 smithi006 ceph-mon[41734]: osdmap e315: 8 total, 8 up, 8 in 2024-03-24T12:16:11.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-24T12:16:11.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi006-81794-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]': finished 2024-03-24T12:16:11.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-24T12:16:11.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:11 smithi006 ceph-mon[41734]: osdmap e316: 8 total, 8 up, 8 in 2024-03-24T12:16:12.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:12 smithi006 ceph-mon[30108]: pgmap v432: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:16:12.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:12 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:12.613 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:12 smithi006 ceph-mon[30108]: osdmap e317: 8 total, 8 up, 8 in 2024-03-24T12:16:12.613 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:12 smithi006 ceph-mon[41734]: pgmap v432: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:16:12.613 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:12 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:12.614 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:12 smithi006 ceph-mon[41734]: osdmap e317: 8 total, 8 up, 8 in 2024-03-24T12:16:12.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:12 smithi090 ceph-mon[33646]: pgmap v432: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:16:12.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:12 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:12.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:12 smithi090 ceph-mon[33646]: osdmap e317: 8 total, 8 up, 8 in 2024-03-24T12:16:12.679 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:16:12 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:16:13.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:13 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:16:13.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:13 smithi090 ceph-mon[33646]: osdmap e318: 8 total, 8 up, 8 in 2024-03-24T12:16:13.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:13 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1685249395' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:13.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:13 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:13.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:13 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:16:13.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:13 smithi006 ceph-mon[30108]: osdmap e318: 8 total, 8 up, 8 in 2024-03-24T12:16:13.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:13 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1685249395' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:13.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:13 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:13.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:13 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:16:13.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:13 smithi006 ceph-mon[41734]: osdmap e318: 8 total, 8 up, 8 in 2024-03-24T12:16:13.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:13 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1685249395' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:13.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:13 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:14.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:14 smithi090 ceph-mon[33646]: pgmap v435: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:16:14.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:14 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]': finished 2024-03-24T12:16:14.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:14 smithi090 ceph-mon[33646]: osdmap e319: 8 total, 8 up, 8 in 2024-03-24T12:16:14.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1685249395' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:14.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:14 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:14.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:14 smithi006 ceph-mon[30108]: pgmap v435: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:16:14.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:14 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]': finished 2024-03-24T12:16:14.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:14 smithi006 ceph-mon[30108]: osdmap e319: 8 total, 8 up, 8 in 2024-03-24T12:16:14.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1685249395' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:14.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:14 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:14.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:14 smithi006 ceph-mon[41734]: pgmap v435: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:16:14.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:14 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi006-81794-59"}]': finished 2024-03-24T12:16:14.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:14 smithi006 ceph-mon[41734]: osdmap e319: 8 total, 8 up, 8 in 2024-03-24T12:16:14.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1685249395' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:14.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:14 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi006-81794-59"}]: dispatch 2024-03-24T12:16:15.015 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:7462ddf6:::1o.RoundTripAppendPP (2990 ms) 2024-03-24T12:16:15.015 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-03-24T12:16:15.015 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3007 ms) 2024-03-24T12:16:15.015 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-03-24T12:16:15.015 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3021 ms) 2024-03-24T12:16:15.015 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-03-24T12:16:15.015 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (2971 ms) 2024-03-24T12:16:15.016 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-03-24T12:16:15.016 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: setting pool EIO 2024-03-24T12:16:15.016 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: max_success 100, min_failed 101 2024-03-24T12:16:15.016 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (4013 ms) 2024-03-24T12:16:15.016 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-03-24T12:16:15.016 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3015 ms) 2024-03-24T12:16:15.016 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (126496 ms total) 2024-03-24T12:16:15.016 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: 2024-03-24T12:16:15.016 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-03-24T12:16:15.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-03-24T12:16:15.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3009 ms) 2024-03-24T12:16:15.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-03-24T12:16:15.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9061 ms) 2024-03-24T12:16:15.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-03-24T12:16:15.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (15329 ms) 2024-03-24T12:16:15.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-03-24T12:16:15.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (2586 ms) 2024-03-24T12:16:15.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (29985 ms total) 2024-03-24T12:16:15.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: 2024-03-24T12:16:15.017 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-03-24T12:16:15.018 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-03-24T12:16:15.018 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3020 ms) 2024-03-24T12:16:15.018 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3020 ms total) 2024-03-24T12:16:15.018 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: 2024-03-24T12:16:15.018 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-03-24T12:16:15.018 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-03-24T12:16:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (13981 ms) 2024-03-24T12:16:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-03-24T12:16:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (7024 ms) 2024-03-24T12:16:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-03-24T12:16:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (6982 ms) 2024-03-24T12:16:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-03-24T12:16:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7037 ms) 2024-03-24T12:16:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-03-24T12:16:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3009 ms) 2024-03-24T12:16:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-03-24T12:16:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7036 ms) 2024-03-24T12:16:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-03-24T12:16:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (7025 ms) 2024-03-24T12:16:15.019 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-03-24T12:16:15.020 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (6916 ms) 2024-03-24T12:16:15.020 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-03-24T12:16:15.020 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7021 ms) 2024-03-24T12:16:15.020 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-03-24T12:16:15.020 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7034 ms) 2024-03-24T12:16:15.020 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-03-24T12:16:15.020 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7034 ms) 2024-03-24T12:16:15.020 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-03-24T12:16:15.020 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7024 ms) 2024-03-24T12:16:15.020 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-03-24T12:16:15.020 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7027 ms) 2024-03-24T12:16:15.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:16:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:16:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:16:16.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:16 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi006-81794-59"}]': finished 2024-03-24T12:16:16.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:16 smithi090 ceph-mon[33646]: osdmap e320: 8 total, 8 up, 8 in 2024-03-24T12:16:16.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:16 smithi090 ceph-mon[33646]: pgmap v438: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:16:16.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:16 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi006-81794-59"}]': finished 2024-03-24T12:16:16.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:16 smithi006 ceph-mon[30108]: osdmap e320: 8 total, 8 up, 8 in 2024-03-24T12:16:16.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:16 smithi006 ceph-mon[30108]: pgmap v438: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:16:16.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:16 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi006-81794-59"}]': finished 2024-03-24T12:16:16.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:16 smithi006 ceph-mon[41734]: osdmap e320: 8 total, 8 up, 8 in 2024-03-24T12:16:16.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:16 smithi006 ceph-mon[41734]: pgmap v438: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:16:17.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:17 smithi090 ceph-mon[33646]: osdmap e321: 8 total, 8 up, 8 in 2024-03-24T12:16:17.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3837382203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi006-81794-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:17.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:17 smithi006 ceph-mon[30108]: osdmap e321: 8 total, 8 up, 8 in 2024-03-24T12:16:17.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3837382203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi006-81794-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:17.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:17 smithi006 ceph-mon[41734]: osdmap e321: 8 total, 8 up, 8 in 2024-03-24T12:16:17.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3837382203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi006-81794-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:18.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3837382203' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi006-81794-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:16:18.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:18 smithi090 ceph-mon[33646]: osdmap e322: 8 total, 8 up, 8 in 2024-03-24T12:16:18.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:18 smithi090 ceph-mon[33646]: pgmap v441: 324 pgs: 324 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-03-24T12:16:18.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3837382203' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi006-81794-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:16:18.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:18 smithi006 ceph-mon[30108]: osdmap e322: 8 total, 8 up, 8 in 2024-03-24T12:16:18.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:18 smithi006 ceph-mon[30108]: pgmap v441: 324 pgs: 324 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-03-24T12:16:18.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3837382203' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi006-81794-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:16:18.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:18 smithi006 ceph-mon[41734]: osdmap e322: 8 total, 8 up, 8 in 2024-03-24T12:16:18.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:18 smithi006 ceph-mon[41734]: pgmap v441: 324 pgs: 324 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-03-24T12:16:19.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:19 smithi090 ceph-mon[33646]: osdmap e323: 8 total, 8 up, 8 in 2024-03-24T12:16:19.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/367787909' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:19.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:19.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/367787909' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:19.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:19.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/367787909' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi006-81794-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:19.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi006-81794-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:19.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:16:19.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:19 smithi006 ceph-mon[30108]: osdmap e323: 8 total, 8 up, 8 in 2024-03-24T12:16:19.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/367787909' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:19.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:19.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/367787909' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:19.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:19.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/367787909' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi006-81794-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:19.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi006-81794-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:19.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:16:19.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:19 smithi006 ceph-mon[41734]: osdmap e323: 8 total, 8 up, 8 in 2024-03-24T12:16:19.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/367787909' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:19.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:19.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/367787909' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:19.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:19.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/367787909' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi006-81794-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:19.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi006-81794-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:19.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:16:20.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:20 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi006-81794-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:20.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:20 smithi090 ceph-mon[33646]: osdmap e324: 8 total, 8 up, 8 in 2024-03-24T12:16:20.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:20 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/367787909' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi006-81794-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:20.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:20 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi006-81794-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:20.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:20 smithi090 ceph-mon[33646]: pgmap v444: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-24T12:16:20.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:20 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi006-81794-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:20.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:20 smithi006 ceph-mon[30108]: osdmap e324: 8 total, 8 up, 8 in 2024-03-24T12:16:20.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:20 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/367787909' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi006-81794-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:20.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:20 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi006-81794-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:20.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:20 smithi006 ceph-mon[30108]: pgmap v444: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-24T12:16:20.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:20 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi006-81794-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:20.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:20 smithi006 ceph-mon[41734]: osdmap e324: 8 total, 8 up, 8 in 2024-03-24T12:16:20.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:20 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/367787909' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi006-81794-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:20.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:20 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi006-81794-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:20.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:20 smithi006 ceph-mon[41734]: pgmap v444: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-24T12:16:21.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:21 smithi090 ceph-mon[33646]: osdmap e325: 8 total, 8 up, 8 in 2024-03-24T12:16:21.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:21 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:21.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:21 smithi006 ceph-mon[30108]: osdmap e325: 8 total, 8 up, 8 in 2024-03-24T12:16:21.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:21 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:21.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:21 smithi006 ceph-mon[41734]: osdmap e325: 8 total, 8 up, 8 in 2024-03-24T12:16:21.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:21 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:22.351 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi006-81794-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi006-81794-61"}]': finished 2024-03-24T12:16:22.351 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:22 smithi090 ceph-mon[33646]: osdmap e326: 8 total, 8 up, 8 in 2024-03-24T12:16:22.351 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:22 smithi090 ceph-mon[33646]: pgmap v447: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:16:22.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi006-81794-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi006-81794-61"}]': finished 2024-03-24T12:16:22.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:22 smithi006 ceph-mon[30108]: osdmap e326: 8 total, 8 up, 8 in 2024-03-24T12:16:22.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:22 smithi006 ceph-mon[30108]: pgmap v447: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:16:22.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi006-81794-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi006-81794-61"}]': finished 2024-03-24T12:16:22.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:22 smithi006 ceph-mon[41734]: osdmap e326: 8 total, 8 up, 8 in 2024-03-24T12:16:22.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:22 smithi006 ceph-mon[41734]: pgmap v447: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:16:22.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:16:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:16:23.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:23 smithi090 ceph-mon[33646]: osdmap e327: 8 total, 8 up, 8 in 2024-03-24T12:16:23.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:23 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:16:23.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:23 smithi006 ceph-mon[30108]: osdmap e327: 8 total, 8 up, 8 in 2024-03-24T12:16:23.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:23 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:16:23.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:23 smithi006 ceph-mon[41734]: osdmap e327: 8 total, 8 up, 8 in 2024-03-24T12:16:23.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:23 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:16:24.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:24.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:24 smithi090 ceph-mon[33646]: osdmap e328: 8 total, 8 up, 8 in 2024-03-24T12:16:24.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:24.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-45"}]: dispatch 2024-03-24T12:16:24.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/367787909' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:24.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-45"}]: dispatch 2024-03-24T12:16:24.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:24.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:24 smithi090 ceph-mon[33646]: pgmap v450: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-24T12:16:24.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:24.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:24 smithi006 ceph-mon[30108]: osdmap e328: 8 total, 8 up, 8 in 2024-03-24T12:16:24.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:24.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-45"}]: dispatch 2024-03-24T12:16:24.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/367787909' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:24.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-45"}]: dispatch 2024-03-24T12:16:24.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:24.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:24 smithi006 ceph-mon[30108]: pgmap v450: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-24T12:16:24.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:24.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:24 smithi006 ceph-mon[41734]: osdmap e328: 8 total, 8 up, 8 in 2024-03-24T12:16:24.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:24.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-45"}]: dispatch 2024-03-24T12:16:24.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/367787909' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:24.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-45"}]: dispatch 2024-03-24T12:16:24.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:24.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:24 smithi006 ceph-mon[41734]: pgmap v450: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-24T12:16:25.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:25 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-45"}]': finished 2024-03-24T12:16:25.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:25 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi006-81794-61"}]': finished 2024-03-24T12:16:25.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:25 smithi090 ceph-mon[33646]: osdmap e329: 8 total, 8 up, 8 in 2024-03-24T12:16:25.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/367787909' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:25.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:25 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:25.460 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:16:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:16:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:16:25.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:25 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-45"}]': finished 2024-03-24T12:16:25.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:25 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi006-81794-61"}]': finished 2024-03-24T12:16:25.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:25 smithi006 ceph-mon[30108]: osdmap e329: 8 total, 8 up, 8 in 2024-03-24T12:16:25.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/367787909' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:25.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:25 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:25.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:25 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-45"}]': finished 2024-03-24T12:16:25.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:25 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi006-81794-61"}]': finished 2024-03-24T12:16:25.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:25 smithi006 ceph-mon[41734]: osdmap e329: 8 total, 8 up, 8 in 2024-03-24T12:16:25.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/367787909' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:25.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:25 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi006-81794-61"}]: dispatch 2024-03-24T12:16:26.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi006-81794-61"}]': finished 2024-03-24T12:16:26.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:26 smithi090 ceph-mon[33646]: osdmap e330: 8 total, 8 up, 8 in 2024-03-24T12:16:26.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4053157825' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:26.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:26 smithi090 ceph-mon[33646]: pgmap v453: 260 pgs: 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:26.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:26.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4053157825' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:26.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:26.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4053157825' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:26.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi006-81794-61"}]': finished 2024-03-24T12:16:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:26 smithi006 ceph-mon[30108]: osdmap e330: 8 total, 8 up, 8 in 2024-03-24T12:16:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4053157825' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:26 smithi006 ceph-mon[30108]: pgmap v453: 260 pgs: 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4053157825' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4053157825' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:26.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi006-81794-61"}]': finished 2024-03-24T12:16:26.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:26 smithi006 ceph-mon[41734]: osdmap e330: 8 total, 8 up, 8 in 2024-03-24T12:16:26.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4053157825' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:26.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:26 smithi006 ceph-mon[41734]: pgmap v453: 260 pgs: 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:26.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:26.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4053157825' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:26.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:26.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4053157825' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:26.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:27.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:27 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:27.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:27 smithi090 ceph-mon[33646]: osdmap e331: 8 total, 8 up, 8 in 2024-03-24T12:16:27.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:27 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4053157825' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi006-81794-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:27.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:27 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:27.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:27 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi006-81794-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:27.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:27 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:27.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:27 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:27.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:27 smithi006 ceph-mon[30108]: osdmap e331: 8 total, 8 up, 8 in 2024-03-24T12:16:27.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:27 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4053157825' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi006-81794-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:27.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:27 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:27.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:27 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi006-81794-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:27.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:27 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:27.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:27 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:27.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:27 smithi006 ceph-mon[41734]: osdmap e331: 8 total, 8 up, 8 in 2024-03-24T12:16:27.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:27 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4053157825' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi006-81794-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:27.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:27 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:27.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:27 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi006-81794-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:27.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:27 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:28.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:16:28.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:28 smithi090 ceph-mon[33646]: pgmap v456: 292 pgs: 2 creating+activating, 30 creating+peering, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:28.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:16:28.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:28 smithi090 ceph-mon[33646]: osdmap e332: 8 total, 8 up, 8 in 2024-03-24T12:16:28.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:16:28.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi006-81794-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi006-81794-62"}]': finished 2024-03-24T12:16:28.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:16:28.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:28 smithi090 ceph-mon[33646]: osdmap e333: 8 total, 8 up, 8 in 2024-03-24T12:16:28.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-47"}]: dispatch 2024-03-24T12:16:28.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-47"}]: dispatch 2024-03-24T12:16:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:16:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:28 smithi006 ceph-mon[30108]: pgmap v456: 292 pgs: 2 creating+activating, 30 creating+peering, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:16:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:28 smithi006 ceph-mon[30108]: osdmap e332: 8 total, 8 up, 8 in 2024-03-24T12:16:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:16:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi006-81794-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi006-81794-62"}]': finished 2024-03-24T12:16:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:16:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:28 smithi006 ceph-mon[30108]: osdmap e333: 8 total, 8 up, 8 in 2024-03-24T12:16:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-47"}]: dispatch 2024-03-24T12:16:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-47"}]: dispatch 2024-03-24T12:16:28.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:16:28.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:28 smithi006 ceph-mon[41734]: pgmap v456: 292 pgs: 2 creating+activating, 30 creating+peering, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:28.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:16:28.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:28 smithi006 ceph-mon[41734]: osdmap e332: 8 total, 8 up, 8 in 2024-03-24T12:16:28.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:16:28.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi006-81794-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi006-81794-62"}]': finished 2024-03-24T12:16:28.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:16:28.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:28 smithi006 ceph-mon[41734]: osdmap e333: 8 total, 8 up, 8 in 2024-03-24T12:16:28.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-47"}]: dispatch 2024-03-24T12:16:28.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-47"}]: dispatch 2024-03-24T12:16:30.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:30 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-47"}]': finished 2024-03-24T12:16:30.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:30 smithi090 ceph-mon[33646]: osdmap e334: 8 total, 8 up, 8 in 2024-03-24T12:16:30.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:30 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-47", "mode": "writeback"}]: dispatch 2024-03-24T12:16:30.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:30 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-47", "mode": "writeback"}]: dispatch 2024-03-24T12:16:30.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:30 smithi090 ceph-mon[33646]: pgmap v459: 300 pgs: 8 unknown, 2 creating+activating, 30 creating+peering, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:30.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:30 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-47"}]': finished 2024-03-24T12:16:30.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:30 smithi006 ceph-mon[30108]: osdmap e334: 8 total, 8 up, 8 in 2024-03-24T12:16:30.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:30 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-47", "mode": "writeback"}]: dispatch 2024-03-24T12:16:30.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:30 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-47", "mode": "writeback"}]: dispatch 2024-03-24T12:16:30.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:30 smithi006 ceph-mon[30108]: pgmap v459: 300 pgs: 8 unknown, 2 creating+activating, 30 creating+peering, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:30.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:30 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-47"}]': finished 2024-03-24T12:16:30.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:30 smithi006 ceph-mon[41734]: osdmap e334: 8 total, 8 up, 8 in 2024-03-24T12:16:30.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:30 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-47", "mode": "writeback"}]: dispatch 2024-03-24T12:16:30.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:30 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-47", "mode": "writeback"}]: dispatch 2024-03-24T12:16:30.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:30 smithi006 ceph-mon[41734]: pgmap v459: 300 pgs: 8 unknown, 2 creating+activating, 30 creating+peering, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:31.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:31 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:31.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:31 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:16:31.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:31 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-47", "mode": "writeback"}]': finished 2024-03-24T12:16:31.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:31 smithi090 ceph-mon[33646]: osdmap e335: 8 total, 8 up, 8 in 2024-03-24T12:16:31.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4053157825' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:31.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:31 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:31.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:16:31.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:31 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:16:31.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:31 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:31.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:31 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:16:31.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:31 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-47", "mode": "writeback"}]': finished 2024-03-24T12:16:31.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:31 smithi006 ceph-mon[30108]: osdmap e335: 8 total, 8 up, 8 in 2024-03-24T12:16:31.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4053157825' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:31.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:31 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:31.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:16:31.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:31 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:16:31.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:31 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:31.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:31 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:16:31.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:31 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-47", "mode": "writeback"}]': finished 2024-03-24T12:16:31.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:31 smithi006 ceph-mon[41734]: osdmap e335: 8 total, 8 up, 8 in 2024-03-24T12:16:31.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4053157825' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:31.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:31 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:31.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:16:31.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:31 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:16:32.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:16:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:16:32.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:32 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62"}]': finished 2024-03-24T12:16:32.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:32 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:16:32.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:32 smithi090 ceph-mon[33646]: osdmap e336: 8 total, 8 up, 8 in 2024-03-24T12:16:32.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:32 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4053157825' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:32.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:32 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:32.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:32 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:16:32.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:32 smithi090 ceph-mon[33646]: pgmap v462: 292 pgs: 2 creating+activating, 30 creating+peering, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:16:32.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:32 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:16:32.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:32 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi006-81794-62"}]': finished 2024-03-24T12:16:32.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:32 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:16:32.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:32 smithi090 ceph-mon[33646]: osdmap e337: 8 total, 8 up, 8 in 2024-03-24T12:16:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:32 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62"}]': finished 2024-03-24T12:16:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:32 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:16:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:32 smithi006 ceph-mon[30108]: osdmap e336: 8 total, 8 up, 8 in 2024-03-24T12:16:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:32 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4053157825' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:32 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:32 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:16:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:32 smithi006 ceph-mon[30108]: pgmap v462: 292 pgs: 2 creating+activating, 30 creating+peering, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:16:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:32 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:16:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:32 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi006-81794-62"}]': finished 2024-03-24T12:16:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:32 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:16:32.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:32 smithi006 ceph-mon[30108]: osdmap e337: 8 total, 8 up, 8 in 2024-03-24T12:16:32.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:32 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi006-81794-62"}]': finished 2024-03-24T12:16:32.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:32 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:16:32.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:32 smithi006 ceph-mon[41734]: osdmap e336: 8 total, 8 up, 8 in 2024-03-24T12:16:32.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:32 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4053157825' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:32.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:32 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi006-81794-62"}]: dispatch 2024-03-24T12:16:32.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:32 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:16:32.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:32 smithi006 ceph-mon[41734]: pgmap v462: 292 pgs: 2 creating+activating, 30 creating+peering, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:16:32.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:32 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:16:32.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:32 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi006-81794-62"}]': finished 2024-03-24T12:16:32.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:32 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:16:32.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:32 smithi006 ceph-mon[41734]: osdmap e337: 8 total, 8 up, 8 in 2024-03-24T12:16:33.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:16:33.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:33 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:16:33.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1438081599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:33.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:33 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:33.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1438081599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:33.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:33 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:33.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1438081599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi006-81794-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:33.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:33 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi006-81794-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:33.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:33 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:16:33.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:33 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:16:33.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:33 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_type","val": "bloom"}]': finished 2024-03-24T12:16:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:16:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:33 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:16:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1438081599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:33 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1438081599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:33 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1438081599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi006-81794-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:33 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi006-81794-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:33.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:33 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:16:33.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:33 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:16:33.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:33 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_type","val": "bloom"}]': finished 2024-03-24T12:16:33.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:16:33.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:33 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:16:33.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1438081599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:33.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:33 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:33.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1438081599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:33.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:33 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:33.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1438081599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi006-81794-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:33.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:33 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi006-81794-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:33.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:33 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:16:33.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:33 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:16:33.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:33 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_type","val": "bloom"}]': finished 2024-03-24T12:16:34.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:34 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi006-81794-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:34.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:34 smithi090 ceph-mon[33646]: osdmap e338: 8 total, 8 up, 8 in 2024-03-24T12:16:34.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:34 smithi090 ceph-mon[33646]: pgmap v465: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-24T12:16:34.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-24T12:16:34.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1438081599' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi006-81794-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:34.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:34 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-24T12:16:34.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:34 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi006-81794-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:34.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:16:34.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:34 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-24T12:16:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:34 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi006-81794-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:34 smithi006 ceph-mon[30108]: osdmap e338: 8 total, 8 up, 8 in 2024-03-24T12:16:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:34 smithi006 ceph-mon[30108]: pgmap v465: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-24T12:16:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-24T12:16:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1438081599' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi006-81794-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:34 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-24T12:16:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:34 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi006-81794-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:16:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:34 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-24T12:16:34.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:34 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi006-81794-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:34.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:34 smithi006 ceph-mon[41734]: osdmap e338: 8 total, 8 up, 8 in 2024-03-24T12:16:34.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:34 smithi006 ceph-mon[41734]: pgmap v465: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-24T12:16:34.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-24T12:16:34.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1438081599' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi006-81794-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:34.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:34 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-24T12:16:34.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:34 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi006-81794-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:34.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:16:34.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:34 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-24T12:16:35.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:35 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-24T12:16:35.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:35 smithi090 ceph-mon[33646]: osdmap e339: 8 total, 8 up, 8 in 2024-03-24T12:16:35.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-24T12:16:35.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi006-81794-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi006-81794-63"}]': finished 2024-03-24T12:16:35.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-24T12:16:35.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:35 smithi090 ceph-mon[33646]: osdmap e340: 8 total, 8 up, 8 in 2024-03-24T12:16:35.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:35 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-24T12:16:35.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-24T12:16:35.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:16:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:16:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:16:35.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:35 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-24T12:16:35.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:35 smithi006 ceph-mon[41734]: osdmap e339: 8 total, 8 up, 8 in 2024-03-24T12:16:35.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-24T12:16:35.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi006-81794-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi006-81794-63"}]': finished 2024-03-24T12:16:35.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-24T12:16:35.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:35 smithi006 ceph-mon[41734]: osdmap e340: 8 total, 8 up, 8 in 2024-03-24T12:16:35.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:35 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-24T12:16:35.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-24T12:16:35.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:35 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-24T12:16:35.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:35 smithi006 ceph-mon[30108]: osdmap e339: 8 total, 8 up, 8 in 2024-03-24T12:16:35.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-24T12:16:35.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi006-81794-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi006-81794-63"}]': finished 2024-03-24T12:16:35.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-24T12:16:35.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:35 smithi006 ceph-mon[30108]: osdmap e340: 8 total, 8 up, 8 in 2024-03-24T12:16:35.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:35 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-24T12:16:35.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-24T12:16:36.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:36 smithi090 ceph-mon[33646]: pgmap v467: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-24T12:16:36.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:36 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:36.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:36 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-24T12:16:36.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:36 smithi090 ceph-mon[33646]: osdmap e341: 8 total, 8 up, 8 in 2024-03-24T12:16:36.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:36 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:36.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:36 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:36.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:36 smithi006 ceph-mon[30108]: pgmap v467: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-24T12:16:36.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:36 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:36.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:36 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-24T12:16:36.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:36 smithi006 ceph-mon[30108]: osdmap e341: 8 total, 8 up, 8 in 2024-03-24T12:16:36.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:36 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:36.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:36 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:36.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:36 smithi006 ceph-mon[41734]: pgmap v467: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-24T12:16:36.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:36 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:36.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:36 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-24T12:16:36.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:36 smithi006 ceph-mon[41734]: osdmap e341: 8 total, 8 up, 8 in 2024-03-24T12:16:36.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:36 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:36.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:36 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:38.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:38 smithi090 ceph-mon[33646]: pgmap v470: 300 pgs: 300 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-24T12:16:38.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:16:38.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:38 smithi090 ceph-mon[33646]: osdmap e342: 8 total, 8 up, 8 in 2024-03-24T12:16:38.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:38 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47"}]: dispatch 2024-03-24T12:16:38.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47"}]: dispatch 2024-03-24T12:16:38.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:38 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1438081599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:38.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:38.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:38 smithi006 ceph-mon[30108]: pgmap v470: 300 pgs: 300 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-24T12:16:38.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:16:38.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:38 smithi006 ceph-mon[30108]: osdmap e342: 8 total, 8 up, 8 in 2024-03-24T12:16:38.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:38 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47"}]: dispatch 2024-03-24T12:16:38.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47"}]: dispatch 2024-03-24T12:16:38.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:38 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1438081599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:38.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:38.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:38 smithi006 ceph-mon[41734]: pgmap v470: 300 pgs: 300 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-24T12:16:38.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:16:38.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:38 smithi006 ceph-mon[41734]: osdmap e342: 8 total, 8 up, 8 in 2024-03-24T12:16:38.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:38 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47"}]: dispatch 2024-03-24T12:16:38.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47"}]: dispatch 2024-03-24T12:16:38.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:38 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1438081599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:38.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:39.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47"}]': finished 2024-03-24T12:16:39.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi006-81794-63"}]': finished 2024-03-24T12:16:39.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:39 smithi090 ceph-mon[33646]: osdmap e343: 8 total, 8 up, 8 in 2024-03-24T12:16:39.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:39 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1438081599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:39.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:39.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:39 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:39.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:39.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:39 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47"}]: dispatch 2024-03-24T12:16:39.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47"}]: dispatch 2024-03-24T12:16:39.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi006-81794-63"}]': finished 2024-03-24T12:16:39.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:39 smithi090 ceph-mon[33646]: osdmap e344: 8 total, 8 up, 8 in 2024-03-24T12:16:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47"}]': finished 2024-03-24T12:16:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi006-81794-63"}]': finished 2024-03-24T12:16:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:39 smithi006 ceph-mon[30108]: osdmap e343: 8 total, 8 up, 8 in 2024-03-24T12:16:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1438081599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47"}]: dispatch 2024-03-24T12:16:39.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47"}]: dispatch 2024-03-24T12:16:39.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi006-81794-63"}]': finished 2024-03-24T12:16:39.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:39 smithi006 ceph-mon[30108]: osdmap e344: 8 total, 8 up, 8 in 2024-03-24T12:16:39.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47"}]': finished 2024-03-24T12:16:39.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi006-81794-63"}]': finished 2024-03-24T12:16:39.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:39 smithi006 ceph-mon[41734]: osdmap e343: 8 total, 8 up, 8 in 2024-03-24T12:16:39.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:39 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1438081599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:39.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi006-81794-63"}]: dispatch 2024-03-24T12:16:39.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:39 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:39.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:39.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:39 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47"}]: dispatch 2024-03-24T12:16:39.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-47"}]: dispatch 2024-03-24T12:16:39.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi006-81794-63"}]': finished 2024-03-24T12:16:39.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:39 smithi006 ceph-mon[41734]: osdmap e344: 8 total, 8 up, 8 in 2024-03-24T12:16:40.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:40 smithi090 ceph-mon[33646]: pgmap v473: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-24T12:16:40.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3704431846' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:40.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:40 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:40.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3704431846' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:40.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:40 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:40.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3704431846' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi006-81794-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:40.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:40 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi006-81794-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:40.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:40 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi006-81794-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:40.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:40 smithi090 ceph-mon[33646]: osdmap e345: 8 total, 8 up, 8 in 2024-03-24T12:16:40.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:40.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3704431846' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi006-81794-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:40.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:40 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:40.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:40 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi006-81794-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[30108]: pgmap v473: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-24T12:16:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3704431846' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3704431846' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3704431846' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi006-81794-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi006-81794-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi006-81794-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[30108]: osdmap e345: 8 total, 8 up, 8 in 2024-03-24T12:16:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:40.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3704431846' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi006-81794-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:40.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:40.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi006-81794-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:40.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[41734]: pgmap v473: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-24T12:16:40.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3704431846' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:40.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:40.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3704431846' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:40.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:40.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3704431846' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi006-81794-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:40.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi006-81794-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:40.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi006-81794-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:40.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[41734]: osdmap e345: 8 total, 8 up, 8 in 2024-03-24T12:16:40.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:40.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3704431846' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi006-81794-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:40.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:40.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:40 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi006-81794-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:42.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:16:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:16:42.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:42 smithi090 ceph-mon[33646]: pgmap v476: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:16:42.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:42 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:16:42.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:42 smithi090 ceph-mon[33646]: osdmap e346: 8 total, 8 up, 8 in 2024-03-24T12:16:42.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:42 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:42.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:42 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:16:42.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:42 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:16:42.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:42 smithi006 ceph-mon[30108]: pgmap v476: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:16:42.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:42 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:16:42.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:42 smithi006 ceph-mon[30108]: osdmap e346: 8 total, 8 up, 8 in 2024-03-24T12:16:42.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:42 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:42.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:42 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:16:42.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:42 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:16:42.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:42 smithi006 ceph-mon[41734]: pgmap v476: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:16:42.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:42 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:16:42.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:42 smithi006 ceph-mon[41734]: osdmap e346: 8 total, 8 up, 8 in 2024-03-24T12:16:42.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:42 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:42.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:42 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:16:42.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:42 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:16:43.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:43 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi006-81794-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi006-81794-64"}]': finished 2024-03-24T12:16:43.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:43 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:16:43.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:43 smithi090 ceph-mon[33646]: osdmap e347: 8 total, 8 up, 8 in 2024-03-24T12:16:43.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:43 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-49"}]: dispatch 2024-03-24T12:16:43.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:43 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-49"}]: dispatch 2024-03-24T12:16:43.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:43 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:16:43.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:43 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi006-81794-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi006-81794-64"}]': finished 2024-03-24T12:16:43.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:43 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:16:43.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:43 smithi006 ceph-mon[30108]: osdmap e347: 8 total, 8 up, 8 in 2024-03-24T12:16:43.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:43 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-49"}]: dispatch 2024-03-24T12:16:43.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:43 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-49"}]: dispatch 2024-03-24T12:16:43.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:43 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:16:43.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:43 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi006-81794-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi006-81794-64"}]': finished 2024-03-24T12:16:43.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:43 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:16:43.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:43 smithi006 ceph-mon[41734]: osdmap e347: 8 total, 8 up, 8 in 2024-03-24T12:16:43.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:43 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-49"}]: dispatch 2024-03-24T12:16:43.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:43 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-49"}]: dispatch 2024-03-24T12:16:43.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:43 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:16:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:44 smithi090 ceph-mon[33646]: pgmap v479: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:16:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:44 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-49"}]': finished 2024-03-24T12:16:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:44 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-49", "mode": "readproxy"}]: dispatch 2024-03-24T12:16:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:44 smithi090 ceph-mon[33646]: osdmap e348: 8 total, 8 up, 8 in 2024-03-24T12:16:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:44 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-49", "mode": "readproxy"}]: dispatch 2024-03-24T12:16:44.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:44 smithi006 ceph-mon[30108]: pgmap v479: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:16:44.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:44 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-49"}]': finished 2024-03-24T12:16:44.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:44 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-49", "mode": "readproxy"}]: dispatch 2024-03-24T12:16:44.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:44 smithi006 ceph-mon[30108]: osdmap e348: 8 total, 8 up, 8 in 2024-03-24T12:16:44.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:44 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-49", "mode": "readproxy"}]: dispatch 2024-03-24T12:16:44.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:44 smithi006 ceph-mon[41734]: pgmap v479: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:16:44.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:44 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-49"}]': finished 2024-03-24T12:16:44.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:44 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-49", "mode": "readproxy"}]: dispatch 2024-03-24T12:16:44.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:44 smithi006 ceph-mon[41734]: osdmap e348: 8 total, 8 up, 8 in 2024-03-24T12:16:44.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:44 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-49", "mode": "readproxy"}]: dispatch 2024-03-24T12:16:45.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:45 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:16:45.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-49", "mode": "readproxy"}]': finished 2024-03-24T12:16:45.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:45 smithi090 ceph-mon[33646]: osdmap e349: 8 total, 8 up, 8 in 2024-03-24T12:16:45.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3704431846' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:45.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:45.461 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:16:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:16:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:16:45.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:45 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:16:45.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-49", "mode": "readproxy"}]': finished 2024-03-24T12:16:45.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:45 smithi006 ceph-mon[30108]: osdmap e349: 8 total, 8 up, 8 in 2024-03-24T12:16:45.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3704431846' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:45.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:45.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:45 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:16:45.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-49", "mode": "readproxy"}]': finished 2024-03-24T12:16:45.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:45 smithi006 ceph-mon[41734]: osdmap e349: 8 total, 8 up, 8 in 2024-03-24T12:16:45.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3704431846' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:45.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:46.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:46 smithi006 ceph-mon[30108]: pgmap v482: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:16:46.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:46 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi006-81794-64"}]': finished 2024-03-24T12:16:46.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:46 smithi006 ceph-mon[30108]: osdmap e350: 8 total, 8 up, 8 in 2024-03-24T12:16:46.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:46 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3704431846' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:46.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:46 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:46.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:46 smithi006 ceph-mon[41734]: pgmap v482: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:16:46.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:46 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi006-81794-64"}]': finished 2024-03-24T12:16:46.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:46 smithi006 ceph-mon[41734]: osdmap e350: 8 total, 8 up, 8 in 2024-03-24T12:16:46.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:46 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3704431846' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:46.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:46 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:46.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:46 smithi090 ceph-mon[33646]: pgmap v482: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:16:46.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:46 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi006-81794-64"}]': finished 2024-03-24T12:16:46.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:46 smithi090 ceph-mon[33646]: osdmap e350: 8 total, 8 up, 8 in 2024-03-24T12:16:46.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:46 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3704431846' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:46.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:46 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi006-81794-64"}]: dispatch 2024-03-24T12:16:47.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:47 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi006-81794-64"}]': finished 2024-03-24T12:16:47.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:47 smithi006 ceph-mon[30108]: osdmap e351: 8 total, 8 up, 8 in 2024-03-24T12:16:47.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi006-81794-65"}]: dispatch 2024-03-24T12:16:47.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi006-81794-65"}]: dispatch 2024-03-24T12:16:47.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi006-81794-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:47.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:47 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi006-81794-64"}]': finished 2024-03-24T12:16:47.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:47 smithi006 ceph-mon[41734]: osdmap e351: 8 total, 8 up, 8 in 2024-03-24T12:16:47.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi006-81794-65"}]: dispatch 2024-03-24T12:16:47.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi006-81794-65"}]: dispatch 2024-03-24T12:16:47.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi006-81794-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:47.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:47 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi006-81794-64"}]': finished 2024-03-24T12:16:47.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:47 smithi090 ceph-mon[33646]: osdmap e351: 8 total, 8 up, 8 in 2024-03-24T12:16:47.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi006-81794-65"}]: dispatch 2024-03-24T12:16:47.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi006-81794-65"}]: dispatch 2024-03-24T12:16:47.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi006-81794-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:48.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:48 smithi006 ceph-mon[30108]: pgmap v485: 292 pgs: 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-03-24T12:16:48.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi006-81794-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:48.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:48 smithi006 ceph-mon[30108]: osdmap e352: 8 total, 8 up, 8 in 2024-03-24T12:16:48.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi006-81794-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi006-81794-65"}]: dispatch 2024-03-24T12:16:48.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:48 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:16:48.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:48 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:16:48.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:48 smithi006 ceph-mon[41734]: pgmap v485: 292 pgs: 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-03-24T12:16:48.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi006-81794-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:48.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:48 smithi006 ceph-mon[41734]: osdmap e352: 8 total, 8 up, 8 in 2024-03-24T12:16:48.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi006-81794-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi006-81794-65"}]: dispatch 2024-03-24T12:16:48.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:48 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:16:48.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:48 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:16:48.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:48 smithi090 ceph-mon[33646]: pgmap v485: 292 pgs: 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-03-24T12:16:48.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi006-81794-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:48.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:48 smithi090 ceph-mon[33646]: osdmap e352: 8 total, 8 up, 8 in 2024-03-24T12:16:48.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi006-81794-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi006-81794-65"}]: dispatch 2024-03-24T12:16:48.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:48 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:16:48.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:48 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:16:49.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:49 smithi090 ceph-mon[33646]: osdmap e353: 8 total, 8 up, 8 in 2024-03-24T12:16:49.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:49 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi006-81794-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi006-81794-65"}]': finished 2024-03-24T12:16:49.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:49 smithi090 ceph-mon[33646]: osdmap e354: 8 total, 8 up, 8 in 2024-03-24T12:16:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:49 smithi006 ceph-mon[30108]: osdmap e353: 8 total, 8 up, 8 in 2024-03-24T12:16:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:49 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi006-81794-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi006-81794-65"}]': finished 2024-03-24T12:16:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:49 smithi006 ceph-mon[30108]: osdmap e354: 8 total, 8 up, 8 in 2024-03-24T12:16:49.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:49 smithi006 ceph-mon[41734]: osdmap e353: 8 total, 8 up, 8 in 2024-03-24T12:16:49.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:49 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi006-81794-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi006-81794-65"}]': finished 2024-03-24T12:16:49.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:49 smithi006 ceph-mon[41734]: osdmap e354: 8 total, 8 up, 8 in 2024-03-24T12:16:50.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:50 smithi090 ceph-mon[33646]: pgmap v488: 292 pgs: 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-03-24T12:16:50.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:50 smithi090 ceph-mon[33646]: osdmap e355: 8 total, 8 up, 8 in 2024-03-24T12:16:50.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:50 smithi006 ceph-mon[30108]: pgmap v488: 292 pgs: 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-03-24T12:16:50.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:50 smithi006 ceph-mon[30108]: osdmap e355: 8 total, 8 up, 8 in 2024-03-24T12:16:50.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:50 smithi006 ceph-mon[41734]: pgmap v488: 292 pgs: 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-03-24T12:16:50.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:50 smithi006 ceph-mon[41734]: osdmap e355: 8 total, 8 up, 8 in 2024-03-24T12:16:51.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:51 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:51.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:51 smithi090 ceph-mon[33646]: osdmap e356: 8 total, 8 up, 8 in 2024-03-24T12:16:51.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:51 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi006-81794-65"}]: dispatch 2024-03-24T12:16:51.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:51 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:51.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:51 smithi006 ceph-mon[30108]: osdmap e356: 8 total, 8 up, 8 in 2024-03-24T12:16:51.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi006-81794-65"}]: dispatch 2024-03-24T12:16:51.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:51 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:51.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:51 smithi006 ceph-mon[41734]: osdmap e356: 8 total, 8 up, 8 in 2024-03-24T12:16:51.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:51 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi006-81794-65"}]: dispatch 2024-03-24T12:16:52.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:16:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:16:52.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:52 smithi090 ceph-mon[33646]: pgmap v491: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:16:52.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:52 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi006-81794-65"}]': finished 2024-03-24T12:16:52.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:52 smithi090 ceph-mon[33646]: osdmap e357: 8 total, 8 up, 8 in 2024-03-24T12:16:52.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:52 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi006-81794-65"}]: dispatch 2024-03-24T12:16:52.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:52 smithi006 ceph-mon[30108]: pgmap v491: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:16:52.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:52 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi006-81794-65"}]': finished 2024-03-24T12:16:52.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:52 smithi006 ceph-mon[30108]: osdmap e357: 8 total, 8 up, 8 in 2024-03-24T12:16:52.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:52 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi006-81794-65"}]: dispatch 2024-03-24T12:16:52.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:52 smithi006 ceph-mon[41734]: pgmap v491: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:16:52.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:52 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi006-81794-65"}]': finished 2024-03-24T12:16:52.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:52 smithi006 ceph-mon[41734]: osdmap e357: 8 total, 8 up, 8 in 2024-03-24T12:16:52.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:52 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi006-81794-65"}]: dispatch 2024-03-24T12:16:53.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:53 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:16:53.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:53 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi006-81794-65"}]': finished 2024-03-24T12:16:53.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:53 smithi090 ceph-mon[33646]: osdmap e358: 8 total, 8 up, 8 in 2024-03-24T12:16:53.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:53 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi006-81794-66"}]: dispatch 2024-03-24T12:16:53.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:53 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi006-81794-66"}]: dispatch 2024-03-24T12:16:53.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:53 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi006-81794-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:53.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:53 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:16:53.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:53 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi006-81794-65"}]': finished 2024-03-24T12:16:53.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:53 smithi006 ceph-mon[30108]: osdmap e358: 8 total, 8 up, 8 in 2024-03-24T12:16:53.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:53 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi006-81794-66"}]: dispatch 2024-03-24T12:16:53.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:53 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi006-81794-66"}]: dispatch 2024-03-24T12:16:53.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:53 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi006-81794-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:53.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:53 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:16:53.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:53 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3925926622' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi006-81794-65"}]': finished 2024-03-24T12:16:53.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:53 smithi006 ceph-mon[41734]: osdmap e358: 8 total, 8 up, 8 in 2024-03-24T12:16:53.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:53 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi006-81794-66"}]: dispatch 2024-03-24T12:16:53.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:53 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi006-81794-66"}]: dispatch 2024-03-24T12:16:53.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:53 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi006-81794-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:16:54.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:54 smithi090 ceph-mon[33646]: pgmap v494: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:54.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi006-81794-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:54.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:54 smithi090 ceph-mon[33646]: osdmap e359: 8 total, 8 up, 8 in 2024-03-24T12:16:54.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi006-81794-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi006-81794-66"}]: dispatch 2024-03-24T12:16:54.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:54 smithi006 ceph-mon[30108]: pgmap v494: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:54.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi006-81794-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:54.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:54 smithi006 ceph-mon[30108]: osdmap e359: 8 total, 8 up, 8 in 2024-03-24T12:16:54.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi006-81794-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi006-81794-66"}]: dispatch 2024-03-24T12:16:54.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:54 smithi006 ceph-mon[41734]: pgmap v494: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:54.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi006-81794-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:16:54.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:54 smithi006 ceph-mon[41734]: osdmap e359: 8 total, 8 up, 8 in 2024-03-24T12:16:54.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi006-81794-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi006-81794-66"}]: dispatch 2024-03-24T12:16:55.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:16:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:16:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:16:55.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:55 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:55.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:55 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:55.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:55 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:16:55.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:55 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49"}]: dispatch 2024-03-24T12:16:55.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:55 smithi006 ceph-mon[30108]: osdmap e360: 8 total, 8 up, 8 in 2024-03-24T12:16:55.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:55 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49"}]: dispatch 2024-03-24T12:16:55.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:55 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:55.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:55 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:55.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:55 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:16:55.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:55 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49"}]: dispatch 2024-03-24T12:16:55.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:55 smithi006 ceph-mon[41734]: osdmap e360: 8 total, 8 up, 8 in 2024-03-24T12:16:55.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:55 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49"}]: dispatch 2024-03-24T12:16:55.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:55 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:55.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:55 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:55.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:55 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:16:55.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:55 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49"}]: dispatch 2024-03-24T12:16:55.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:55 smithi090 ceph-mon[33646]: osdmap e360: 8 total, 8 up, 8 in 2024-03-24T12:16:55.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:55 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49"}]: dispatch 2024-03-24T12:16:56.829 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:56 smithi090 ceph-mon[33646]: pgmap v497: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:56.829 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:56 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:16:56.830 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:56 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi006-81794-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi006-81794-66"}]': finished 2024-03-24T12:16:56.830 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:56 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49"}]': finished 2024-03-24T12:16:56.830 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:56 smithi090 ceph-mon[33646]: osdmap e361: 8 total, 8 up, 8 in 2024-03-24T12:16:56.830 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:56 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:56.830 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:56 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:56.830 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:56 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49"}]: dispatch 2024-03-24T12:16:56.830 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:56 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49"}]: dispatch 2024-03-24T12:16:56.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:56 smithi006 ceph-mon[30108]: pgmap v497: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:56.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:56 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:16:56.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:56 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi006-81794-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi006-81794-66"}]': finished 2024-03-24T12:16:56.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:56 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49"}]': finished 2024-03-24T12:16:56.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:56 smithi006 ceph-mon[30108]: osdmap e361: 8 total, 8 up, 8 in 2024-03-24T12:16:56.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:56 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:56.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:56 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:56.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:56 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49"}]: dispatch 2024-03-24T12:16:56.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:56 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49"}]: dispatch 2024-03-24T12:16:56.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:56 smithi006 ceph-mon[41734]: pgmap v497: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:56.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:56 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:16:56.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:56 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi006-81794-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi006-81794-66"}]': finished 2024-03-24T12:16:56.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:56 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49"}]': finished 2024-03-24T12:16:56.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:56 smithi006 ceph-mon[41734]: osdmap e361: 8 total, 8 up, 8 in 2024-03-24T12:16:56.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:56 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:56.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:56 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:16:56.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:56 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49"}]: dispatch 2024-03-24T12:16:56.963 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:56 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-49"}]: dispatch 2024-03-24T12:16:57.225 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP63:head 2024-03-24T12:16:57.225 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:5d165639:::164:head 2024-03-24T12:16:57.225 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:f43765fc:::165:head 2024-03-24T12:16:57.225 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:b4c720e9:::166:head 2024-03-24T12:16:57.225 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:e694b040:::167:head 2024-03-24T12:16:57.225 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:afa38db2:::168:head 2024-03-24T12:16:57.225 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:77ba9f53:::169:head 2024-03-24T12:16:57.225 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:87495034:::170:head 2024-03-24T12:16:57.225 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:7c96bf0e:::171:head 2024-03-24T12:16:57.226 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:dbe346cc:::172:head 2024-03-24T12:16:57.226 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:e943ec24:::173:head 2024-03-24T12:16:57.226 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:f97a9c0c:::174:head 2024-03-24T12:16:57.226 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:6f26e74d:::175:head 2024-03-24T12:16:57.226 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:4f95e106:::176:head 2024-03-24T12:16:57.226 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:0e6f2f8f:::177:head 2024-03-24T12:16:57.226 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:05db05f1:::178:head 2024-03-24T12:16:57.226 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:38a78d66:::179:head 2024-03-24T12:16:57.226 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:d095610b:::180:head 2024-03-24T12:16:57.226 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:a1a9d709:::181:head 2024-03-24T12:16:57.226 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:1e5d39db:::182:head 2024-03-24T12:16:57.226 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:f7df4fb9:::183:head 2024-03-24T12:16:57.226 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:03a7f161:::184:head 2024-03-24T12:16:57.226 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:ba70721e:::185:head 2024-03-24T12:16:57.226 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:28e5662d:::186:head 2024-03-24T12:16:57.227 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:973d52de:::187:head 2024-03-24T12:16:57.227 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:4303eb1c:::188:head 2024-03-24T12:16:57.227 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:b990b48e:::189:head 2024-03-24T12:16:57.227 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:29b8165b:::190:head 2024-03-24T12:16:57.227 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:3547f197:::191:head 2024-03-24T12:16:57.227 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:7e260936:::192:head 2024-03-24T12:16:57.227 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:1abec7b1:::193:head 2024-03-24T12:16:57.227 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:10fdda93:::194:head 2024-03-24T12:16:57.227 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:15817eea:::195:head 2024-03-24T12:16:57.227 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:770bab57:::196:head 2024-03-24T12:16:57.227 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:ed9e13e7:::197:head 2024-03-24T12:16:57.227 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:71471a8f:::198:head 2024-03-24T12:16:57.227 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: checking for 277:10fb1d02:::199:head 2024-03-24T12:16:57.227 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (7777 ms) 2024-03-24T12:16:57.227 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-03-24T12:16:57.228 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711282572,0 2024-03-24T12:16:57.228 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: first is 1711282572 2024-03-24T12:16:57.228 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711282572,0 2024-03-24T12:16:57.228 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711282572,0 2024-03-24T12:16:57.228 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711282572,1711282574,1711282575,0 2024-03-24T12:16:57.228 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711282572,1711282574,1711282575,0 2024-03-24T12:16:57.228 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711282572,1711282574,1711282575,0 2024-03-24T12:16:57.228 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711282572,1711282574,1711282575,1711282577,1711282578,0 2024-03-24T12:16:57.228 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711282572,1711282574,1711282575,1711282577,1711282578,0 2024-03-24T12:16:57.228 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711282572,1711282574,1711282575,1711282577,1711282578,0 2024-03-24T12:16:57.228 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711282572,1711282574,1711282575,1711282577,1711282578,1711282580,1711282581,0 2024-03-24T12:16:57.228 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711282572,1711282574,1711282575,1711282577,1711282578,1711282580,1711282581,0 2024-03-24T12:16:57.228 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711282572,1711282574,1711282575,1711282577,1711282578,1711282580,1711282581,0 2024-03-24T12:16:57.228 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711282575,1711282577,1711282578,1711282580,1711282581,1711282583,1711282584,0 2024-03-24T12:16:57.228 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: first now 1711282575, trimmed 2024-03-24T12:16:57.229 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (21119 ms) 2024-03-24T12:16:57.229 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-03-24T12:16:57.229 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: foo0 2024-03-24T12:16:57.229 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-03-24T12:16:57.229 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14053 ms) 2024-03-24T12:16:57.229 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-03-24T12:16:57.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:57 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:16:57.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:57 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:16:57.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:57 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:16:57.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:57 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:16:57.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:57 smithi090 ceph-mon[33646]: osdmap e362: 8 total, 8 up, 8 in 2024-03-24T12:16:57.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:57 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:16:57.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:57 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:16:57.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:57 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:16:57.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:57 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:16:57.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:57 smithi006 ceph-mon[30108]: osdmap e362: 8 total, 8 up, 8 in 2024-03-24T12:16:57.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:57 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:16:57.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:57 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:16:57.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:57 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:16:57.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:57 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:16:57.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:57 smithi006 ceph-mon[41734]: osdmap e362: 8 total, 8 up, 8 in 2024-03-24T12:16:58.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:58 smithi090 ceph-mon[33646]: pgmap v500: 300 pgs: 8 creating+peering, 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:58.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:58 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:58.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:58 smithi090 ceph-mon[33646]: osdmap e363: 8 total, 8 up, 8 in 2024-03-24T12:16:58.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:58 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi006-81794-66"}]: dispatch 2024-03-24T12:16:58.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:58 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:58.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:16:58 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:58.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:58 smithi006 ceph-mon[30108]: pgmap v500: 300 pgs: 8 creating+peering, 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:58.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:58 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:58.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:58 smithi006 ceph-mon[30108]: osdmap e363: 8 total, 8 up, 8 in 2024-03-24T12:16:58.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:58 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi006-81794-66"}]: dispatch 2024-03-24T12:16:58.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:58 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:58.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:16:58 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:58.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:58 smithi006 ceph-mon[41734]: pgmap v500: 300 pgs: 8 creating+peering, 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:16:58.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:58 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:16:58.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:58 smithi006 ceph-mon[41734]: osdmap e363: 8 total, 8 up, 8 in 2024-03-24T12:16:58.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:58 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi006-81794-66"}]: dispatch 2024-03-24T12:16:58.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:58 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:16:58.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:16:58 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:00.233 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ 2 2024-03-24T12:17:00.234 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3030 ms) 2024-03-24T12:17:00.234 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-03-24T12:17:00.234 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7050 ms) 2024-03-24T12:17:00.234 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-03-24T12:17:00.234 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7029 ms) 2024-03-24T12:17:00.234 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-03-24T12:17:00.234 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7022 ms) 2024-03-24T12:17:00.234 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-03-24T12:17:00.234 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7050 ms) 2024-03-24T12:17:00.234 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-03-24T12:17:00.234 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7019 ms) 2024-03-24T12:17:00.234 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-03-24T12:17:00.234 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7016 ms) 2024-03-24T12:17:00.235 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139367 ms total) 2024-03-24T12:17:00.235 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: 2024-03-24T12:17:00.235 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-03-24T12:17:00.235 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (298868 ms total) 2024-03-24T12:17:00.235 INFO:tasks.workunit.client.0.smithi006.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-03-24T12:17:00.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:00 smithi090 ceph-mon[33646]: pgmap v503: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:17:00.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:00 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi006-81794-66"}]': finished 2024-03-24T12:17:00.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:00 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:17:00.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:00 smithi090 ceph-mon[33646]: osdmap e364: 8 total, 8 up, 8 in 2024-03-24T12:17:00.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:00 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi006-81794-66"}]: dispatch 2024-03-24T12:17:00.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:00 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:17:00.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:00 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:17:00.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:00 smithi006 ceph-mon[30108]: pgmap v503: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:17:00.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:00 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi006-81794-66"}]': finished 2024-03-24T12:17:00.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:00 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:17:00.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:00 smithi006 ceph-mon[30108]: osdmap e364: 8 total, 8 up, 8 in 2024-03-24T12:17:00.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:00 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi006-81794-66"}]: dispatch 2024-03-24T12:17:00.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:00 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:17:00.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:00 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:17:00.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:00 smithi006 ceph-mon[41734]: pgmap v503: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:17:00.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:00 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi006-81794-66"}]': finished 2024-03-24T12:17:00.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:00 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:17:00.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:00 smithi006 ceph-mon[41734]: osdmap e364: 8 total, 8 up, 8 in 2024-03-24T12:17:00.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:00 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi006-81794-66"}]: dispatch 2024-03-24T12:17:00.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:00 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:17:00.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:00 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:17:01.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi006-81794-66"}]': finished 2024-03-24T12:17:01.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:17:01.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:01 smithi090 ceph-mon[33646]: osdmap e365: 8 total, 8 up, 8 in 2024-03-24T12:17:01.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-51"}]: dispatch 2024-03-24T12:17:01.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-51"}]: dispatch 2024-03-24T12:17:01.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-51"}]': finished 2024-03-24T12:17:01.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi006-81794-66"}]': finished 2024-03-24T12:17:01.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:17:01.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:01 smithi006 ceph-mon[30108]: osdmap e365: 8 total, 8 up, 8 in 2024-03-24T12:17:01.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-51"}]: dispatch 2024-03-24T12:17:01.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-51"}]: dispatch 2024-03-24T12:17:01.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-51"}]': finished 2024-03-24T12:17:01.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2319348419' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi006-81794-66"}]': finished 2024-03-24T12:17:01.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:17:01.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:01 smithi006 ceph-mon[41734]: osdmap e365: 8 total, 8 up, 8 in 2024-03-24T12:17:01.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-51"}]: dispatch 2024-03-24T12:17:01.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-51"}]: dispatch 2024-03-24T12:17:01.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-51"}]': finished 2024-03-24T12:17:02.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:17:02 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:17:02.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:02 smithi090 ceph-mon[33646]: pgmap v506: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:17:02.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:02 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-51", "mode": "writeback"}]: dispatch 2024-03-24T12:17:02.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:02 smithi090 ceph-mon[33646]: osdmap e366: 8 total, 8 up, 8 in 2024-03-24T12:17:02.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:02 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-51", "mode": "writeback"}]: dispatch 2024-03-24T12:17:02.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:02 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:17:02.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:02 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-51", "mode": "writeback"}]': finished 2024-03-24T12:17:02.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:02 smithi090 ceph-mon[33646]: osdmap e367: 8 total, 8 up, 8 in 2024-03-24T12:17:02.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:02 smithi006 ceph-mon[30108]: pgmap v506: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:17:02.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:02 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-51", "mode": "writeback"}]: dispatch 2024-03-24T12:17:02.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:02 smithi006 ceph-mon[30108]: osdmap e366: 8 total, 8 up, 8 in 2024-03-24T12:17:02.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:02 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-51", "mode": "writeback"}]: dispatch 2024-03-24T12:17:02.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:02 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:17:02.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:02 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-51", "mode": "writeback"}]': finished 2024-03-24T12:17:02.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:02 smithi006 ceph-mon[30108]: osdmap e367: 8 total, 8 up, 8 in 2024-03-24T12:17:02.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:02 smithi006 ceph-mon[41734]: pgmap v506: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:17:02.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:02 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-51", "mode": "writeback"}]: dispatch 2024-03-24T12:17:02.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:02 smithi006 ceph-mon[41734]: osdmap e366: 8 total, 8 up, 8 in 2024-03-24T12:17:02.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:02 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-51", "mode": "writeback"}]: dispatch 2024-03-24T12:17:02.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:02 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:17:02.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:02 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-51", "mode": "writeback"}]': finished 2024-03-24T12:17:02.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:02 smithi006 ceph-mon[41734]: osdmap e367: 8 total, 8 up, 8 in 2024-03-24T12:17:03.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:17:03.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:03 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:17:03.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:03 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:17:03.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:03 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:17:03.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:17:03.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:03 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:17:03.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:03 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:17:03.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:03 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:17:03.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:17:03.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:03 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:17:03.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:03 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:17:03.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:03 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:17:04.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:04 smithi090 ceph-mon[33646]: pgmap v509: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:17:04.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:04 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:17:04.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:04 smithi090 ceph-mon[33646]: osdmap e368: 8 total, 8 up, 8 in 2024-03-24T12:17:04.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:04 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:17:04.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:04 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:17:04.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:04 smithi006 ceph-mon[30108]: pgmap v509: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:17:04.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:04 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:17:04.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:04 smithi006 ceph-mon[30108]: osdmap e368: 8 total, 8 up, 8 in 2024-03-24T12:17:04.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:04 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:17:04.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:04 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:17:04.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:04 smithi006 ceph-mon[41734]: pgmap v509: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:17:04.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:04 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:17:04.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:04 smithi006 ceph-mon[41734]: osdmap e368: 8 total, 8 up, 8 in 2024-03-24T12:17:04.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:04 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:17:04.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:04 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:17:05.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:05 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:17:05.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:05 smithi090 ceph-mon[33646]: osdmap e369: 8 total, 8 up, 8 in 2024-03-24T12:17:05.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:05 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:17:05.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:05 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:17:05.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:05 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:17:05.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:05 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_type","val": "bloom"}]': finished 2024-03-24T12:17:05.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:17:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:17:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:17:05.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:05 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:17:05.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:05 smithi006 ceph-mon[30108]: osdmap e369: 8 total, 8 up, 8 in 2024-03-24T12:17:05.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:05 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:17:05.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:05 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:17:05.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:05 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:17:05.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:05 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_type","val": "bloom"}]': finished 2024-03-24T12:17:05.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:05 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:17:05.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:05 smithi006 ceph-mon[41734]: osdmap e369: 8 total, 8 up, 8 in 2024-03-24T12:17:05.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:05 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:17:05.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:05 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:17:05.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:05 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:17:05.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:05 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "hit_set_type","val": "bloom"}]': finished 2024-03-24T12:17:06.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:06 smithi090 ceph-mon[33646]: pgmap v512: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:17:06.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:06 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-24T12:17:06.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:06 smithi090 ceph-mon[33646]: osdmap e370: 8 total, 8 up, 8 in 2024-03-24T12:17:06.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:06 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-24T12:17:06.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:06 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:17:06.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:06 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-24T12:17:06.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:06 smithi090 ceph-mon[33646]: osdmap e371: 8 total, 8 up, 8 in 2024-03-24T12:17:06.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:06 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-24T12:17:06.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:06 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-24T12:17:06.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:06 smithi006 ceph-mon[30108]: pgmap v512: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:17:06.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:06 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-24T12:17:06.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:06 smithi006 ceph-mon[30108]: osdmap e370: 8 total, 8 up, 8 in 2024-03-24T12:17:06.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:06 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-24T12:17:06.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:06 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:17:06.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:06 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-24T12:17:06.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:06 smithi006 ceph-mon[30108]: osdmap e371: 8 total, 8 up, 8 in 2024-03-24T12:17:06.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:06 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-24T12:17:06.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:06 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-24T12:17:06.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:06 smithi006 ceph-mon[41734]: pgmap v512: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:17:06.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:06 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-24T12:17:06.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:06 smithi006 ceph-mon[41734]: osdmap e370: 8 total, 8 up, 8 in 2024-03-24T12:17:06.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:06 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-24T12:17:06.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:06 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:17:06.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:06 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-24T12:17:06.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:06 smithi006 ceph-mon[41734]: osdmap e371: 8 total, 8 up, 8 in 2024-03-24T12:17:06.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:06 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-24T12:17:06.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:06 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-24T12:17:08.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:08 smithi090 ceph-mon[33646]: pgmap v515: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T12:17:08.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "target_max_objects","val": "1"}]': finished 2024-03-24T12:17:08.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:08 smithi090 ceph-mon[33646]: osdmap e372: 8 total, 8 up, 8 in 2024-03-24T12:17:08.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:08 smithi006 ceph-mon[30108]: pgmap v515: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T12:17:08.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "target_max_objects","val": "1"}]': finished 2024-03-24T12:17:08.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:08 smithi006 ceph-mon[30108]: osdmap e372: 8 total, 8 up, 8 in 2024-03-24T12:17:08.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:08 smithi006 ceph-mon[41734]: pgmap v515: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T12:17:08.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-51","var": "target_max_objects","val": "1"}]': finished 2024-03-24T12:17:08.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:08 smithi006 ceph-mon[41734]: osdmap e372: 8 total, 8 up, 8 in 2024-03-24T12:17:09.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:09 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-24T12:17:09.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:09 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-24T12:17:09.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:09 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-24T12:17:10.509 INFO:tasks.workunit.client.0.smithi006.stdout: started 0x55ac4adb43a0 2024-03-24T12:17:10.509 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb44d0 2024-03-24T12:17:10.510 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb4600 2024-03-24T12:17:10.510 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb4730 2024-03-24T12:17:10.510 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb4e10 2024-03-24T12:17:10.510 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb4970 2024-03-24T12:17:10.510 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb4ac0 2024-03-24T12:17:10.510 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb4bd0 2024-03-24T12:17:10.510 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb4d00 2024-03-24T12:17:10.510 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb53e0 2024-03-24T12:17:10.510 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb4f40 2024-03-24T12:17:10.510 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb5070 2024-03-24T12:17:10.510 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb5180 2024-03-24T12:17:10.510 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb52b0 2024-03-24T12:17:10.510 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb59d0 2024-03-24T12:17:10.510 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb5530 2024-03-24T12:17:10.510 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb5640 2024-03-24T12:17:10.511 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb57b0 2024-03-24T12:17:10.511 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb58c0 2024-03-24T12:17:10.511 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb5fc0 2024-03-24T12:17:10.511 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb5b20 2024-03-24T12:17:10.511 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb5c50 2024-03-24T12:17:10.511 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb5d60 2024-03-24T12:17:10.511 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb5eb0 2024-03-24T12:17:10.511 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb6590 2024-03-24T12:17:10.511 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb60f0 2024-03-24T12:17:10.511 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb6260 2024-03-24T12:17:10.511 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb6370 2024-03-24T12:17:10.511 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb66a0 2024-03-24T12:17:10.511 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb6b50 2024-03-24T12:17:10.511 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb6c80 2024-03-24T12:17:10.512 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb67d0 2024-03-24T12:17:10.512 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb6920 2024-03-24T12:17:10.512 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb7250 2024-03-24T12:17:10.512 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb6db0 2024-03-24T12:17:10.512 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb6ec0 2024-03-24T12:17:10.512 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb6ff0 2024-03-24T12:17:10.512 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb7120 2024-03-24T12:17:10.512 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb7840 2024-03-24T12:17:10.512 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb73a0 2024-03-24T12:17:10.512 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb74d0 2024-03-24T12:17:10.512 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb75e0 2024-03-24T12:17:10.512 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb7710 2024-03-24T12:17:10.512 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb7e10 2024-03-24T12:17:10.512 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb7970 2024-03-24T12:17:10.513 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb7aa0 2024-03-24T12:17:10.513 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb7bd0 2024-03-24T12:17:10.513 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb7d00 2024-03-24T12:17:10.513 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb83e0 2024-03-24T12:17:10.513 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb7f40 2024-03-24T12:17:10.513 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb8070 2024-03-24T12:17:10.513 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb81c0 2024-03-24T12:17:10.513 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb8510 2024-03-24T12:17:10.513 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb89a0 2024-03-24T12:17:10.513 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb8af0 2024-03-24T12:17:10.513 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb8620 2024-03-24T12:17:10.513 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb8770 2024-03-24T12:17:10.513 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb82d0 2024-03-24T12:17:10.513 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb8c40 2024-03-24T12:17:10.513 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb8d70 2024-03-24T12:17:10.513 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb8880 2024-03-24T12:17:10.514 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb8ee0 2024-03-24T12:17:10.514 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb9390 2024-03-24T12:17:10.514 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb94c0 2024-03-24T12:17:10.514 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb9a90 2024-03-24T12:17:10.514 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb9030 2024-03-24T12:17:10.514 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb9160 2024-03-24T12:17:10.514 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb95f0 2024-03-24T12:17:10.514 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb9720 2024-03-24T12:17:10.514 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb9850 2024-03-24T12:17:10.514 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb9980 2024-03-24T12:17:10.514 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adba060 2024-03-24T12:17:10.514 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb9bc0 2024-03-24T12:17:10.514 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb9cf0 2024-03-24T12:17:10.514 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb9e00 2024-03-24T12:17:10.515 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb9f50 2024-03-24T12:17:10.515 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adb9270 2024-03-24T12:17:10.515 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adba190 2024-03-24T12:17:10.515 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adba670 2024-03-24T12:17:10.515 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adba7a0 2024-03-24T12:17:10.515 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adba2a0 2024-03-24T12:17:10.515 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adba3f0 2024-03-24T12:17:10.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:10 smithi090 ceph-mon[33646]: pgmap v517: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T12:17:10.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:10 smithi006 ceph-mon[30108]: pgmap v517: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T12:17:10.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:10 smithi006 ceph-mon[41734]: pgmap v517: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T12:17:12.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:12 smithi090 ceph-mon[33646]: pgmap v518: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-24T12:17:12.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:17:12 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:17:12.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:12 smithi006 ceph-mon[30108]: pgmap v518: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-24T12:17:12.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:12 smithi006 ceph-mon[41734]: pgmap v518: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-24T12:17:13.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:13 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:17:13.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:13 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:17:13.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:13 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:17:14.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:14 smithi090 ceph-mon[33646]: pgmap v519: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-24T12:17:14.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:14 smithi006 ceph-mon[30108]: pgmap v519: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-24T12:17:14.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:14 smithi006 ceph-mon[41734]: pgmap v519: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-24T12:17:15.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:17:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:17:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:17:16.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:16 smithi090 ceph-mon[33646]: pgmap v520: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-24T12:17:16.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:16 smithi006 ceph-mon[30108]: pgmap v520: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-24T12:17:16.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:16 smithi006 ceph-mon[41734]: pgmap v520: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-24T12:17:17.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:17.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:17.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:17.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:17.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:17.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:18.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:18 smithi090 ceph-mon[33646]: pgmap v521: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:17:18.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:17:18.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51"}]: dispatch 2024-03-24T12:17:18.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:18 smithi090 ceph-mon[33646]: osdmap e373: 8 total, 8 up, 8 in 2024-03-24T12:17:18.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51"}]: dispatch 2024-03-24T12:17:18.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:18 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:17:18.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:18 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:17:18.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:18 smithi006 ceph-mon[30108]: pgmap v521: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:17:18.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:17:18.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51"}]: dispatch 2024-03-24T12:17:18.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:18 smithi006 ceph-mon[30108]: osdmap e373: 8 total, 8 up, 8 in 2024-03-24T12:17:18.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51"}]: dispatch 2024-03-24T12:17:18.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:18 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:17:18.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:18 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:17:18.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:18 smithi006 ceph-mon[41734]: pgmap v521: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:17:18.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:17:18.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51"}]: dispatch 2024-03-24T12:17:18.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:18 smithi006 ceph-mon[41734]: osdmap e373: 8 total, 8 up, 8 in 2024-03-24T12:17:18.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51"}]: dispatch 2024-03-24T12:17:18.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:18 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:17:18.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:18 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:17:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51"}]': finished 2024-03-24T12:17:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:19 smithi090 ceph-mon[33646]: osdmap e374: 8 total, 8 up, 8 in 2024-03-24T12:17:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51"}]: dispatch 2024-03-24T12:17:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51"}]: dispatch 2024-03-24T12:17:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51"}]': finished 2024-03-24T12:17:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:19 smithi006 ceph-mon[30108]: osdmap e374: 8 total, 8 up, 8 in 2024-03-24T12:17:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51"}]: dispatch 2024-03-24T12:17:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51"}]: dispatch 2024-03-24T12:17:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51"}]': finished 2024-03-24T12:17:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:19 smithi006 ceph-mon[41734]: osdmap e374: 8 total, 8 up, 8 in 2024-03-24T12:17:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:19.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:19.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51"}]: dispatch 2024-03-24T12:17:19.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-51"}]: dispatch 2024-03-24T12:17:20.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:20 smithi090 ceph-mon[33646]: pgmap v524: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:17:20.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:20 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-24T12:17:20.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:20 smithi090 ceph-mon[33646]: osdmap e375: 8 total, 8 up, 8 in 2024-03-24T12:17:20.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:20 smithi006 ceph-mon[30108]: pgmap v524: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:17:20.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:20 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-24T12:17:20.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:20 smithi006 ceph-mon[30108]: osdmap e375: 8 total, 8 up, 8 in 2024-03-24T12:17:20.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:20 smithi006 ceph-mon[41734]: pgmap v524: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:17:20.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:20 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-24T12:17:20.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:20 smithi006 ceph-mon[41734]: osdmap e375: 8 total, 8 up, 8 in 2024-03-24T12:17:21.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:21 smithi090 ceph-mon[33646]: osdmap e376: 8 total, 8 up, 8 in 2024-03-24T12:17:21.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:21.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:21 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:21.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:21 smithi006 ceph-mon[30108]: osdmap e376: 8 total, 8 up, 8 in 2024-03-24T12:17:21.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:21.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:21 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:21.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:21 smithi006 ceph-mon[41734]: osdmap e376: 8 total, 8 up, 8 in 2024-03-24T12:17:21.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:21.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:21 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:22.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:17:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:17:22.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:22 smithi090 ceph-mon[33646]: pgmap v527: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:17:22.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:22 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:17:22.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:17:22.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:22 smithi090 ceph-mon[33646]: osdmap e377: 8 total, 8 up, 8 in 2024-03-24T12:17:22.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:22.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:22.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-53"}]: dispatch 2024-03-24T12:17:22.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-53"}]: dispatch 2024-03-24T12:17:22.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:22 smithi006 ceph-mon[30108]: pgmap v527: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:17:22.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:22 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:17:22.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:17:22.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:22 smithi006 ceph-mon[30108]: osdmap e377: 8 total, 8 up, 8 in 2024-03-24T12:17:22.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:22.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:22.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-53"}]: dispatch 2024-03-24T12:17:22.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-53"}]: dispatch 2024-03-24T12:17:22.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:22 smithi006 ceph-mon[41734]: pgmap v527: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:17:22.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:22 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:17:22.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:17:22.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:22 smithi006 ceph-mon[41734]: osdmap e377: 8 total, 8 up, 8 in 2024-03-24T12:17:22.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:22.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:22.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-53"}]: dispatch 2024-03-24T12:17:22.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-53"}]: dispatch 2024-03-24T12:17:23.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:23 smithi090 ceph-mon[33646]: osdmap e378: 8 total, 8 up, 8 in 2024-03-24T12:17:23.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:23 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:17:23.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:23 smithi006 ceph-mon[30108]: osdmap e378: 8 total, 8 up, 8 in 2024-03-24T12:17:23.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:23 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:17:23.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:23 smithi006 ceph-mon[41734]: osdmap e378: 8 total, 8 up, 8 in 2024-03-24T12:17:23.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:23 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:17:24.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:24 smithi090 ceph-mon[33646]: pgmap v530: 260 pgs: 260 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-24T12:17:24.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:24 smithi090 ceph-mon[33646]: osdmap e379: 8 total, 8 up, 8 in 2024-03-24T12:17:24.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:24.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:24.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:24 smithi006 ceph-mon[30108]: pgmap v530: 260 pgs: 260 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-24T12:17:24.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:24 smithi006 ceph-mon[30108]: osdmap e379: 8 total, 8 up, 8 in 2024-03-24T12:17:24.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:24.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:24.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:24 smithi006 ceph-mon[41734]: pgmap v530: 260 pgs: 260 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-24T12:17:24.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:24 smithi006 ceph-mon[41734]: osdmap e379: 8 total, 8 up, 8 in 2024-03-24T12:17:24.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:24.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:25.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:25 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:17:25.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:25 smithi090 ceph-mon[33646]: osdmap e380: 8 total, 8 up, 8 in 2024-03-24T12:17:25.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:17:25.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:25.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:25 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:25.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-55"}]: dispatch 2024-03-24T12:17:25.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:25 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-55"}]: dispatch 2024-03-24T12:17:25.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:17:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:17:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:17:25.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:25 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:17:25.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:25 smithi006 ceph-mon[30108]: osdmap e380: 8 total, 8 up, 8 in 2024-03-24T12:17:25.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:17:25.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:25.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:25 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:25.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-55"}]: dispatch 2024-03-24T12:17:25.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:25 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-55"}]: dispatch 2024-03-24T12:17:25.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:25 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:17:25.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:25 smithi006 ceph-mon[41734]: osdmap e380: 8 total, 8 up, 8 in 2024-03-24T12:17:25.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:17:25.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:25.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:25 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:25.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-55"}]: dispatch 2024-03-24T12:17:25.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:25 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-55"}]: dispatch 2024-03-24T12:17:26.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:26 smithi090 ceph-mon[33646]: pgmap v533: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:17:26.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:26 smithi090 ceph-mon[33646]: osdmap e381: 8 total, 8 up, 8 in 2024-03-24T12:17:26.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:26 smithi006 ceph-mon[30108]: pgmap v533: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:17:26.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:26 smithi006 ceph-mon[30108]: osdmap e381: 8 total, 8 up, 8 in 2024-03-24T12:17:26.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:26 smithi006 ceph-mon[41734]: pgmap v533: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:17:26.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:26 smithi006 ceph-mon[41734]: osdmap e381: 8 total, 8 up, 8 in 2024-03-24T12:17:27.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:27 smithi090 ceph-mon[33646]: osdmap e382: 8 total, 8 up, 8 in 2024-03-24T12:17:27.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:27 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:27.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:27 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:27.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:27 smithi006 ceph-mon[30108]: osdmap e382: 8 total, 8 up, 8 in 2024-03-24T12:17:27.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:27 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:27.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:27 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:27.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:27 smithi006 ceph-mon[41734]: osdmap e382: 8 total, 8 up, 8 in 2024-03-24T12:17:27.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:27 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:27.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:27 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:28.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:28 smithi090 ceph-mon[33646]: pgmap v536: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:17:28.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:28 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:17:28.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:17:28.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:28 smithi090 ceph-mon[33646]: osdmap e383: 8 total, 8 up, 8 in 2024-03-24T12:17:28.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:28.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:28.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-57"}]: dispatch 2024-03-24T12:17:28.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-57"}]: dispatch 2024-03-24T12:17:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:28 smithi006 ceph-mon[30108]: pgmap v536: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:17:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:28 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:17:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:17:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:28 smithi006 ceph-mon[30108]: osdmap e383: 8 total, 8 up, 8 in 2024-03-24T12:17:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-57"}]: dispatch 2024-03-24T12:17:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-57"}]: dispatch 2024-03-24T12:17:28.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:28 smithi006 ceph-mon[41734]: pgmap v536: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:17:28.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:28 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:17:28.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:17:28.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:28 smithi006 ceph-mon[41734]: osdmap e383: 8 total, 8 up, 8 in 2024-03-24T12:17:28.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:28.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:28.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-57"}]: dispatch 2024-03-24T12:17:28.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-57"}]: dispatch 2024-03-24T12:17:29.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:29 smithi006 ceph-mon[30108]: osdmap e384: 8 total, 8 up, 8 in 2024-03-24T12:17:29.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:29 smithi006 ceph-mon[41734]: osdmap e384: 8 total, 8 up, 8 in 2024-03-24T12:17:29.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:29 smithi090 ceph-mon[33646]: osdmap e384: 8 total, 8 up, 8 in 2024-03-24T12:17:30.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:30 smithi006 ceph-mon[30108]: pgmap v539: 260 pgs: 260 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:17:30.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:30 smithi006 ceph-mon[30108]: osdmap e385: 8 total, 8 up, 8 in 2024-03-24T12:17:30.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:30 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:30.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:30 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:30.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:30 smithi006 ceph-mon[41734]: pgmap v539: 260 pgs: 260 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:17:30.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:30 smithi006 ceph-mon[41734]: osdmap e385: 8 total, 8 up, 8 in 2024-03-24T12:17:30.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:30 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:30.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:30 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:30 smithi090 ceph-mon[33646]: pgmap v539: 260 pgs: 260 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:17:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:30 smithi090 ceph-mon[33646]: osdmap e385: 8 total, 8 up, 8 in 2024-03-24T12:17:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:30 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:30 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:31.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:31 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:17:31.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:31 smithi006 ceph-mon[30108]: osdmap e386: 8 total, 8 up, 8 in 2024-03-24T12:17:31.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:17:31.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:31.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:31 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:31.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-59"}]: dispatch 2024-03-24T12:17:31.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:31 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-59"}]: dispatch 2024-03-24T12:17:31.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:31 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:17:31.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:31 smithi006 ceph-mon[41734]: osdmap e386: 8 total, 8 up, 8 in 2024-03-24T12:17:31.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:17:31.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:31.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:31 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:31.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-59"}]: dispatch 2024-03-24T12:17:31.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:31 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-59"}]: dispatch 2024-03-24T12:17:31.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:31 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:17:31.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:31 smithi090 ceph-mon[33646]: osdmap e386: 8 total, 8 up, 8 in 2024-03-24T12:17:31.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:17:31.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:31.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:31 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:31.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-59"}]: dispatch 2024-03-24T12:17:31.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:31 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-59"}]: dispatch 2024-03-24T12:17:32.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:32 smithi090 ceph-mon[33646]: pgmap v542: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:17:32.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:32 smithi090 ceph-mon[33646]: osdmap e387: 8 total, 8 up, 8 in 2024-03-24T12:17:32.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:17:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:17:32.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:32 smithi006 ceph-mon[30108]: pgmap v542: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:17:32.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:32 smithi006 ceph-mon[30108]: osdmap e387: 8 total, 8 up, 8 in 2024-03-24T12:17:32.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:32 smithi006 ceph-mon[41734]: pgmap v542: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:17:32.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:32 smithi006 ceph-mon[41734]: osdmap e387: 8 total, 8 up, 8 in 2024-03-24T12:17:33.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:33 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:17:33.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:33 smithi090 ceph-mon[33646]: osdmap e388: 8 total, 8 up, 8 in 2024-03-24T12:17:33.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:33.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:33 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:33.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:33 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:17:33.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:33 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:17:33.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:33 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:17:33.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:33 smithi006 ceph-mon[30108]: osdmap e388: 8 total, 8 up, 8 in 2024-03-24T12:17:33.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:33.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:33 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:33.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:33 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:17:33.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:33 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:17:33.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:33 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:17:33.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:33 smithi006 ceph-mon[41734]: osdmap e388: 8 total, 8 up, 8 in 2024-03-24T12:17:33.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:33.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:33 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:33.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:33 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:17:33.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:33 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:17:34.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:34 smithi090 ceph-mon[33646]: pgmap v545: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-24T12:17:34.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:34 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:17:34.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:34 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:17:34.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:34 smithi090 ceph-mon[33646]: osdmap e389: 8 total, 8 up, 8 in 2024-03-24T12:17:34.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:17:34.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:17:34.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:34 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:17:34.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:34 smithi006 ceph-mon[30108]: pgmap v545: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-24T12:17:34.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:34 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:17:34.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:34 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:17:34.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:34 smithi006 ceph-mon[30108]: osdmap e389: 8 total, 8 up, 8 in 2024-03-24T12:17:34.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:17:34.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:17:34.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:34 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:17:34.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:34 smithi006 ceph-mon[41734]: pgmap v545: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-24T12:17:34.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:34 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:17:34.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:34 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:17:34.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:34 smithi006 ceph-mon[41734]: osdmap e389: 8 total, 8 up, 8 in 2024-03-24T12:17:34.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:17:34.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:17:34.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:34 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:17:35.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:17:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:17:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:17:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:17:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:35 smithi006 ceph-mon[30108]: osdmap e390: 8 total, 8 up, 8 in 2024-03-24T12:17:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:35 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:35 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-61"}]: dispatch 2024-03-24T12:17:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-61"}]: dispatch 2024-03-24T12:17:35.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:17:35.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:35 smithi006 ceph-mon[41734]: osdmap e390: 8 total, 8 up, 8 in 2024-03-24T12:17:35.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:35 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:35.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:35.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:35 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-61"}]: dispatch 2024-03-24T12:17:35.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-61"}]: dispatch 2024-03-24T12:17:35.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:17:35.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:35 smithi090 ceph-mon[33646]: osdmap e390: 8 total, 8 up, 8 in 2024-03-24T12:17:35.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:35 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:35.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:17:35.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:35 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-61"}]: dispatch 2024-03-24T12:17:35.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-61"}]: dispatch 2024-03-24T12:17:36.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:36 smithi090 ceph-mon[33646]: pgmap v548: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-24T12:17:36.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:36 smithi090 ceph-mon[33646]: osdmap e391: 8 total, 8 up, 8 in 2024-03-24T12:17:36.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:36 smithi006 ceph-mon[30108]: pgmap v548: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-24T12:17:36.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:36 smithi006 ceph-mon[30108]: osdmap e391: 8 total, 8 up, 8 in 2024-03-24T12:17:36.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:36 smithi006 ceph-mon[41734]: pgmap v548: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-24T12:17:36.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:36 smithi006 ceph-mon[41734]: osdmap e391: 8 total, 8 up, 8 in 2024-03-24T12:17:37.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:37 smithi090 ceph-mon[33646]: osdmap e392: 8 total, 8 up, 8 in 2024-03-24T12:17:37.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:37.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:37 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:37.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:37 smithi006 ceph-mon[30108]: osdmap e392: 8 total, 8 up, 8 in 2024-03-24T12:17:37.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:37.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:37 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:37.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:37 smithi006 ceph-mon[41734]: osdmap e392: 8 total, 8 up, 8 in 2024-03-24T12:17:37.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:37.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:37 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:17:38.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:38 smithi090 ceph-mon[33646]: pgmap v551: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:17:38.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:17:38.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:38 smithi090 ceph-mon[33646]: osdmap e393: 8 total, 8 up, 8 in 2024-03-24T12:17:38.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:38 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:17:38.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:38 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:17:38.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:17:38.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:38 smithi006 ceph-mon[30108]: pgmap v551: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:17:38.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:17:38.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:38 smithi006 ceph-mon[30108]: osdmap e393: 8 total, 8 up, 8 in 2024-03-24T12:17:38.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:38 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:17:38.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:38 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:17:38.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:17:38.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:38 smithi006 ceph-mon[41734]: pgmap v551: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:17:38.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:17:38.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:38 smithi006 ceph-mon[41734]: osdmap e393: 8 total, 8 up, 8 in 2024-03-24T12:17:38.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:38 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:17:38.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:38 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:17:38.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:17:39.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:17:39.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:39 smithi090 ceph-mon[33646]: osdmap e394: 8 total, 8 up, 8 in 2024-03-24T12:17:39.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:17:39.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:39 smithi006 ceph-mon[30108]: osdmap e394: 8 total, 8 up, 8 in 2024-03-24T12:17:39.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:17:39.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:39 smithi006 ceph-mon[41734]: osdmap e394: 8 total, 8 up, 8 in 2024-03-24T12:17:40.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:40 smithi090 ceph-mon[33646]: pgmap v554: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:17:40.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:40 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:17:40.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:40 smithi090 ceph-mon[33646]: osdmap e395: 8 total, 8 up, 8 in 2024-03-24T12:17:40.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:40 smithi006 ceph-mon[30108]: pgmap v554: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:17:40.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:40 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:17:40.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:40 smithi006 ceph-mon[30108]: osdmap e395: 8 total, 8 up, 8 in 2024-03-24T12:17:40.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:40 smithi006 ceph-mon[41734]: pgmap v554: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:17:40.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:40 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:17:40.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:40 smithi006 ceph-mon[41734]: osdmap e395: 8 total, 8 up, 8 in 2024-03-24T12:17:41.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:41 smithi090 ceph-mon[33646]: osdmap e396: 8 total, 8 up, 8 in 2024-03-24T12:17:41.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:41 smithi006 ceph-mon[30108]: osdmap e396: 8 total, 8 up, 8 in 2024-03-24T12:17:41.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:41 smithi006 ceph-mon[41734]: osdmap e396: 8 total, 8 up, 8 in 2024-03-24T12:17:42.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:42 smithi090 ceph-mon[33646]: pgmap v557: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:17:42.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:42 smithi090 ceph-mon[33646]: osdmap e397: 8 total, 8 up, 8 in 2024-03-24T12:17:42.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:17:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:17:42.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:42 smithi006 ceph-mon[30108]: pgmap v557: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:17:42.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:42 smithi006 ceph-mon[30108]: osdmap e397: 8 total, 8 up, 8 in 2024-03-24T12:17:42.862 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:42 smithi006 ceph-mon[41734]: pgmap v557: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:17:42.862 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:42 smithi006 ceph-mon[41734]: osdmap e397: 8 total, 8 up, 8 in 2024-03-24T12:17:43.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:43 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:17:43.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:43 smithi090 ceph-mon[33646]: osdmap e398: 8 total, 8 up, 8 in 2024-03-24T12:17:43.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:43 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:17:43.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:43 smithi006 ceph-mon[30108]: osdmap e398: 8 total, 8 up, 8 in 2024-03-24T12:17:43.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:43 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:17:43.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:43 smithi006 ceph-mon[41734]: osdmap e398: 8 total, 8 up, 8 in 2024-03-24T12:17:44.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:44 smithi090 ceph-mon[33646]: pgmap v560: 292 pgs: 292 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-03-24T12:17:44.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:44 smithi006 ceph-mon[30108]: pgmap v560: 292 pgs: 292 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-03-24T12:17:44.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:44 smithi006 ceph-mon[41734]: pgmap v560: 292 pgs: 292 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-03-24T12:17:45.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:17:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:17:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:17:46.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:46 smithi090 ceph-mon[33646]: pgmap v561: 292 pgs: 292 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-03-24T12:17:46.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:46 smithi006 ceph-mon[30108]: pgmap v561: 292 pgs: 292 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-03-24T12:17:46.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:46 smithi006 ceph-mon[41734]: pgmap v561: 292 pgs: 292 active+clean; 8.4 MiB data, 669 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-03-24T12:17:48.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:48 smithi090 ceph-mon[33646]: pgmap v562: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-24T12:17:48.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:48 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:17:48.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:48 smithi006 ceph-mon[30108]: pgmap v562: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-24T12:17:48.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:48 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:17:48.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:48 smithi006 ceph-mon[41734]: pgmap v562: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-24T12:17:48.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:48 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:17:50.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:50 smithi090 ceph-mon[33646]: pgmap v563: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-03-24T12:17:50.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:50 smithi006 ceph-mon[30108]: pgmap v563: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-03-24T12:17:50.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:50 smithi006 ceph-mon[41734]: pgmap v563: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-03-24T12:17:52.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:51 smithi090 ceph-mon[33646]: osdmap e399: 8 total, 8 up, 8 in 2024-03-24T12:17:52.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:51 smithi090 ceph-mon[33646]: pgmap v565: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 713 B/s rd, 0 B/s wr, 0 op/s 2024-03-24T12:17:52.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:51 smithi006 ceph-mon[30108]: osdmap e399: 8 total, 8 up, 8 in 2024-03-24T12:17:52.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:51 smithi006 ceph-mon[30108]: pgmap v565: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 713 B/s rd, 0 B/s wr, 0 op/s 2024-03-24T12:17:52.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:51 smithi006 ceph-mon[41734]: osdmap e399: 8 total, 8 up, 8 in 2024-03-24T12:17:52.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:51 smithi006 ceph-mon[41734]: pgmap v565: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 713 B/s rd, 0 B/s wr, 0 op/s 2024-03-24T12:17:52.875 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:17:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:17:53.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:52 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:17:53.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:52 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:17:53.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:52 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:17:54.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:53 smithi090 ceph-mon[33646]: osdmap e400: 8 total, 8 up, 8 in 2024-03-24T12:17:54.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:53 smithi090 ceph-mon[33646]: pgmap v567: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:17:54.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:53 smithi006 ceph-mon[30108]: osdmap e400: 8 total, 8 up, 8 in 2024-03-24T12:17:54.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:53 smithi006 ceph-mon[30108]: pgmap v567: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:17:54.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:53 smithi006 ceph-mon[41734]: osdmap e400: 8 total, 8 up, 8 in 2024-03-24T12:17:54.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:53 smithi006 ceph-mon[41734]: pgmap v567: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:17:55.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:17:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:17:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:17:56.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:56 smithi006 ceph-mon[30108]: pgmap v568: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T12:17:56.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:56 smithi006 ceph-mon[41734]: pgmap v568: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T12:17:56.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:56 smithi090 ceph-mon[33646]: pgmap v568: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T12:17:58.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:58 smithi006 ceph-mon[30108]: pgmap v569: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:17:58.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:58 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:17:58.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:58 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:17:58.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:58 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:17:58.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:17:58 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:17:58.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:58 smithi006 ceph-mon[41734]: pgmap v569: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:17:58.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:58 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:17:58.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:58 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:17:58.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:58 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:17:58.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:17:58 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:17:58.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:58 smithi090 ceph-mon[33646]: pgmap v569: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:17:58.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:58 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:17:58.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:58 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:17:58.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:58 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:17:58.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:17:58 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:18:00.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:00 smithi090 ceph-mon[33646]: pgmap v570: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:18:00.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:00 smithi006 ceph-mon[30108]: pgmap v570: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:18:00.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:00 smithi006 ceph-mon[41734]: pgmap v570: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:18:02.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:01 smithi090 ceph-mon[33646]: osdmap e401: 8 total, 8 up, 8 in 2024-03-24T12:18:02.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:01 smithi090 ceph-mon[33646]: pgmap v572: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 990 B/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:18:02.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:01 smithi006 ceph-mon[30108]: osdmap e401: 8 total, 8 up, 8 in 2024-03-24T12:18:02.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:01 smithi006 ceph-mon[30108]: pgmap v572: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 990 B/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:18:02.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:01 smithi006 ceph-mon[41734]: osdmap e401: 8 total, 8 up, 8 in 2024-03-24T12:18:02.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:01 smithi006 ceph-mon[41734]: pgmap v572: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 990 B/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:18:02.875 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:18:02 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:18:03.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:02 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:18:03.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:02 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:18:03.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:02 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:18:04.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:03 smithi090 ceph-mon[33646]: pgmap v573: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:18:04.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:03 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:18:04.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:03 smithi006 ceph-mon[30108]: pgmap v573: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:18:04.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:03 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:18:04.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:03 smithi006 ceph-mon[41734]: pgmap v573: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:18:04.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:03 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:18:05.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:04 smithi090 ceph-mon[33646]: osdmap e402: 8 total, 8 up, 8 in 2024-03-24T12:18:05.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:04 smithi006 ceph-mon[30108]: osdmap e402: 8 total, 8 up, 8 in 2024-03-24T12:18:05.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:04 smithi006 ceph-mon[41734]: osdmap e402: 8 total, 8 up, 8 in 2024-03-24T12:18:05.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:18:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:18:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:18:06.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:05 smithi090 ceph-mon[33646]: pgmap v575: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T12:18:06.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:05 smithi006 ceph-mon[30108]: pgmap v575: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T12:18:06.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:05 smithi006 ceph-mon[41734]: pgmap v575: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T12:18:08.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:08 smithi090 ceph-mon[33646]: pgmap v576: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-24T12:18:08.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:08 smithi006 ceph-mon[30108]: pgmap v576: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-24T12:18:08.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:08 smithi006 ceph-mon[41734]: pgmap v576: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-24T12:18:10.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:10 smithi090 ceph-mon[33646]: pgmap v577: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T12:18:10.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:10 smithi006 ceph-mon[30108]: pgmap v577: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T12:18:10.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:10 smithi006 ceph-mon[41734]: pgmap v577: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T12:18:12.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:11 smithi090 ceph-mon[33646]: osdmap e403: 8 total, 8 up, 8 in 2024-03-24T12:18:12.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:11 smithi090 ceph-mon[33646]: pgmap v579: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:18:12.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:11 smithi006 ceph-mon[30108]: osdmap e403: 8 total, 8 up, 8 in 2024-03-24T12:18:12.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:11 smithi006 ceph-mon[30108]: pgmap v579: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:18:12.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:11 smithi006 ceph-mon[41734]: osdmap e403: 8 total, 8 up, 8 in 2024-03-24T12:18:12.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:11 smithi006 ceph-mon[41734]: pgmap v579: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:18:12.878 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:18:12 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:18:13.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:12 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:18:13.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:12 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:18:13.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:12 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:18:14.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:13 smithi090 ceph-mon[33646]: pgmap v580: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:18:14.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:13 smithi006 ceph-mon[30108]: pgmap v580: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:18:14.497 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:13 smithi006 ceph-mon[41734]: pgmap v580: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:18:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:14 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-63"}]: dispatch 2024-03-24T12:18:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:14 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-63"}]: dispatch 2024-03-24T12:18:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:14 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-63"}]: dispatch 2024-03-24T12:18:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:14 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-63"}]: dispatch 2024-03-24T12:18:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:14 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-63"}]: dispatch 2024-03-24T12:18:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:14 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-63"}]: dispatch 2024-03-24T12:18:15.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:18:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:18:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:18:16.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:15 smithi090 ceph-mon[33646]: osdmap e404: 8 total, 8 up, 8 in 2024-03-24T12:18:16.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:15 smithi090 ceph-mon[33646]: pgmap v582: 260 pgs: 260 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:18:16.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:15 smithi006 ceph-mon[30108]: osdmap e404: 8 total, 8 up, 8 in 2024-03-24T12:18:16.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:15 smithi006 ceph-mon[30108]: pgmap v582: 260 pgs: 260 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:18:16.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:15 smithi006 ceph-mon[41734]: osdmap e404: 8 total, 8 up, 8 in 2024-03-24T12:18:16.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:15 smithi006 ceph-mon[41734]: pgmap v582: 260 pgs: 260 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:18:17.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:16 smithi090 ceph-mon[33646]: osdmap e405: 8 total, 8 up, 8 in 2024-03-24T12:18:17.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:16 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:17.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:16 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:17.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:16 smithi006 ceph-mon[30108]: osdmap e405: 8 total, 8 up, 8 in 2024-03-24T12:18:17.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:16 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:17.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:16 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:17.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:16 smithi006 ceph-mon[41734]: osdmap e405: 8 total, 8 up, 8 in 2024-03-24T12:18:17.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:16 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:17.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:16 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:18:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:17 smithi090 ceph-mon[33646]: osdmap e406: 8 total, 8 up, 8 in 2024-03-24T12:18:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:18:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:18:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:18:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:17 smithi090 ceph-mon[33646]: pgmap v585: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:18:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:18:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:17 smithi006 ceph-mon[30108]: osdmap e406: 8 total, 8 up, 8 in 2024-03-24T12:18:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:18:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:18:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:18:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:17 smithi006 ceph-mon[30108]: pgmap v585: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:18:18.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:18:18.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:17 smithi006 ceph-mon[41734]: osdmap e406: 8 total, 8 up, 8 in 2024-03-24T12:18:18.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:18:18.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:18:18.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:18:18.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:17 smithi006 ceph-mon[41734]: pgmap v585: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:18:19.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:18:19.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:18 smithi090 ceph-mon[33646]: osdmap e407: 8 total, 8 up, 8 in 2024-03-24T12:18:19.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:18 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:18:19.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:18:19.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:18 smithi006 ceph-mon[30108]: osdmap e407: 8 total, 8 up, 8 in 2024-03-24T12:18:19.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:18 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:18:19.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:18:19.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:18 smithi006 ceph-mon[41734]: osdmap e407: 8 total, 8 up, 8 in 2024-03-24T12:18:19.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:18 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:18:20.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:19 smithi090 ceph-mon[33646]: osdmap e408: 8 total, 8 up, 8 in 2024-03-24T12:18:20.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:19 smithi090 ceph-mon[33646]: pgmap v588: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:18:20.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:19 smithi006 ceph-mon[30108]: osdmap e408: 8 total, 8 up, 8 in 2024-03-24T12:18:20.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:19 smithi006 ceph-mon[30108]: pgmap v588: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:18:20.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:19 smithi006 ceph-mon[41734]: osdmap e408: 8 total, 8 up, 8 in 2024-03-24T12:18:20.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:19 smithi006 ceph-mon[41734]: pgmap v588: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:18:21.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:20 smithi006 ceph-mon[30108]: osdmap e409: 8 total, 8 up, 8 in 2024-03-24T12:18:21.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:20 smithi006 ceph-mon[41734]: osdmap e409: 8 total, 8 up, 8 in 2024-03-24T12:18:21.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:20 smithi090 ceph-mon[33646]: osdmap e409: 8 total, 8 up, 8 in 2024-03-24T12:18:22.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:21 smithi006 ceph-mon[30108]: osdmap e410: 8 total, 8 up, 8 in 2024-03-24T12:18:22.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:21 smithi006 ceph-mon[30108]: pgmap v591: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:18:22.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:21 smithi006 ceph-mon[41734]: osdmap e410: 8 total, 8 up, 8 in 2024-03-24T12:18:22.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:21 smithi006 ceph-mon[41734]: pgmap v591: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:18:22.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:21 smithi090 ceph-mon[33646]: osdmap e410: 8 total, 8 up, 8 in 2024-03-24T12:18:22.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:21 smithi090 ceph-mon[33646]: pgmap v591: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:18:22.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:18:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:18:23.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:22 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:18:23.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:22 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:18:23.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:22 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:18:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:23 smithi006 ceph-mon[30108]: pgmap v592: 292 pgs: 292 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-03-24T12:18:24.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:23 smithi006 ceph-mon[41734]: pgmap v592: 292 pgs: 292 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-03-24T12:18:24.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:23 smithi090 ceph-mon[33646]: pgmap v592: 292 pgs: 292 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-03-24T12:18:25.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:18:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:18:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:18:26.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:26 smithi090 ceph-mon[33646]: pgmap v593: 292 pgs: 292 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-03-24T12:18:26.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:26 smithi090 ceph-mon[33646]: osdmap e411: 8 total, 8 up, 8 in 2024-03-24T12:18:26.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:26 smithi006 ceph-mon[30108]: pgmap v593: 292 pgs: 292 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-03-24T12:18:26.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:26 smithi006 ceph-mon[30108]: osdmap e411: 8 total, 8 up, 8 in 2024-03-24T12:18:26.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:26 smithi006 ceph-mon[41734]: pgmap v593: 292 pgs: 292 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-03-24T12:18:26.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:26 smithi006 ceph-mon[41734]: osdmap e411: 8 total, 8 up, 8 in 2024-03-24T12:18:28.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:28 smithi090 ceph-mon[33646]: pgmap v595: 292 pgs: 292 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-03-24T12:18:28.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:28 smithi006 ceph-mon[30108]: pgmap v595: 292 pgs: 292 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-03-24T12:18:28.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:28 smithi006 ceph-mon[41734]: pgmap v595: 292 pgs: 292 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-03-24T12:18:30.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:30 smithi090 ceph-mon[33646]: pgmap v596: 292 pgs: 292 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-24T12:18:30.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:30 smithi006 ceph-mon[30108]: pgmap v596: 292 pgs: 292 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-24T12:18:30.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:30 smithi006 ceph-mon[41734]: pgmap v596: 292 pgs: 292 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-24T12:18:31.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:31.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:31 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:31.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-65"}]: dispatch 2024-03-24T12:18:31.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:31 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-65"}]: dispatch 2024-03-24T12:18:31.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:31.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:31 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:31.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-65"}]: dispatch 2024-03-24T12:18:31.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:31 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-65"}]: dispatch 2024-03-24T12:18:31.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:31.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:31 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:31.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-65"}]: dispatch 2024-03-24T12:18:31.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:31 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-65"}]: dispatch 2024-03-24T12:18:32.525 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adba520 2024-03-24T12:18:32.525 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adba8f0 2024-03-24T12:18:32.525 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbadd0 2024-03-24T12:18:32.525 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbaf70 2024-03-24T12:18:32.526 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbb540 2024-03-24T12:18:32.526 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbaa20 2024-03-24T12:18:32.526 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbab50 2024-03-24T12:18:32.526 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbb0a0 2024-03-24T12:18:32.526 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbb1d0 2024-03-24T12:18:32.526 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbb300 2024-03-24T12:18:32.526 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbb430 2024-03-24T12:18:32.526 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbbb10 2024-03-24T12:18:32.526 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbb670 2024-03-24T12:18:32.526 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbb7a0 2024-03-24T12:18:32.526 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbb8b0 2024-03-24T12:18:32.526 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbb9e0 2024-03-24T12:18:32.526 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbc0e0 2024-03-24T12:18:32.527 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbbc40 2024-03-24T12:18:32.527 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbbd90 2024-03-24T12:18:32.527 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbbea0 2024-03-24T12:18:32.527 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbbfd0 2024-03-24T12:18:32.527 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbc6d0 2024-03-24T12:18:32.527 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbc230 2024-03-24T12:18:32.527 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbc360 2024-03-24T12:18:32.527 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbac80 2024-03-24T12:18:32.527 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbc470 2024-03-24T12:18:32.527 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbc5c0 2024-03-24T12:18:32.527 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbc820 2024-03-24T12:18:32.527 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbccd0 2024-03-24T12:18:32.527 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbce00 2024-03-24T12:18:32.527 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbc950 2024-03-24T12:18:32.527 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbcaa0 2024-03-24T12:18:32.527 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbd3d0 2024-03-24T12:18:32.528 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbcf30 2024-03-24T12:18:32.528 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbd060 2024-03-24T12:18:32.528 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbd190 2024-03-24T12:18:32.528 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbd500 2024-03-24T12:18:32.528 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbda00 2024-03-24T12:18:32.528 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbdb80 2024-03-24T12:18:32.528 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaa460 2024-03-24T12:18:32.528 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adaa590 2024-03-24T12:18:32.528 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbe150 2024-03-24T12:18:32.528 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbdcb0 2024-03-24T12:18:32.528 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbddc0 2024-03-24T12:18:32.528 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbdef0 2024-03-24T12:18:32.528 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbe020 2024-03-24T12:18:32.528 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbe720 2024-03-24T12:18:32.528 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbe280 2024-03-24T12:18:32.529 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbe3d0 2024-03-24T12:18:32.529 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbe500 2024-03-24T12:18:32.529 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbe850 2024-03-24T12:18:32.529 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbed30 2024-03-24T12:18:32.529 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbee60 2024-03-24T12:18:32.529 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbd650 2024-03-24T12:18:32.529 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbd780 2024-03-24T12:18:32.529 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbf410 2024-03-24T12:18:32.529 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbef70 2024-03-24T12:18:32.529 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbf0a0 2024-03-24T12:18:32.529 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbf1d0 2024-03-24T12:18:32.529 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbf300 2024-03-24T12:18:32.529 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbf9e0 2024-03-24T12:18:32.529 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbf540 2024-03-24T12:18:32.529 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbf690 2024-03-24T12:18:32.529 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbf7c0 2024-03-24T12:18:32.530 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbfb10 2024-03-24T12:18:32.530 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbffa0 2024-03-24T12:18:32.530 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc00d0 2024-03-24T12:18:32.530 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbe960 2024-03-24T12:18:32.530 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbea90 2024-03-24T12:18:32.530 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc06a0 2024-03-24T12:18:32.530 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc0200 2024-03-24T12:18:32.530 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc0330 2024-03-24T12:18:32.530 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc0460 2024-03-24T12:18:32.530 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc0590 2024-03-24T12:18:32.530 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc0c70 2024-03-24T12:18:32.530 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc07d0 2024-03-24T12:18:32.530 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc0900 2024-03-24T12:18:32.530 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc0a30 2024-03-24T12:18:32.530 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc0b40 2024-03-24T12:18:32.531 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc1220 2024-03-24T12:18:32.531 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc0d80 2024-03-24T12:18:32.531 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc0eb0 2024-03-24T12:18:32.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:32 smithi090 ceph-mon[33646]: pgmap v597: 292 pgs: 292 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-24T12:18:32.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:32 smithi090 ceph-mon[33646]: osdmap e412: 8 total, 8 up, 8 in 2024-03-24T12:18:32.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:18:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:18:32.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:32 smithi006 ceph-mon[30108]: pgmap v597: 292 pgs: 292 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-24T12:18:32.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:32 smithi006 ceph-mon[30108]: osdmap e412: 8 total, 8 up, 8 in 2024-03-24T12:18:32.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:32 smithi006 ceph-mon[41734]: pgmap v597: 292 pgs: 292 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-24T12:18:32.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:32 smithi006 ceph-mon[41734]: osdmap e412: 8 total, 8 up, 8 in 2024-03-24T12:18:33.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:33 smithi090 ceph-mon[33646]: osdmap e413: 8 total, 8 up, 8 in 2024-03-24T12:18:33.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:33.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:33 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:33.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:33 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:18:33.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:33 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:18:33.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:33 smithi090 ceph-mon[33646]: osdmap e414: 8 total, 8 up, 8 in 2024-03-24T12:18:33.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:33 smithi006 ceph-mon[30108]: osdmap e413: 8 total, 8 up, 8 in 2024-03-24T12:18:33.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:33.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:33 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:33.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:33 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:18:33.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:33 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:18:33.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:33 smithi006 ceph-mon[30108]: osdmap e414: 8 total, 8 up, 8 in 2024-03-24T12:18:33.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:33 smithi006 ceph-mon[41734]: osdmap e413: 8 total, 8 up, 8 in 2024-03-24T12:18:33.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:33.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:33 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:33.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:33 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:18:33.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:33 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:18:33.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:33 smithi006 ceph-mon[41734]: osdmap e414: 8 total, 8 up, 8 in 2024-03-24T12:18:34.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:34 smithi090 ceph-mon[33646]: pgmap v600: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:18:34.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:18:34.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:34 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:18:34.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:18:34.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:34 smithi090 ceph-mon[33646]: osdmap e415: 8 total, 8 up, 8 in 2024-03-24T12:18:34.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:34 smithi006 ceph-mon[30108]: pgmap v600: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:18:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:18:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:34 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:18:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:18:34.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:34 smithi006 ceph-mon[30108]: osdmap e415: 8 total, 8 up, 8 in 2024-03-24T12:18:34.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:34 smithi006 ceph-mon[41734]: pgmap v600: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:18:34.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:18:34.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:34 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:18:34.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:18:34.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:34 smithi006 ceph-mon[41734]: osdmap e415: 8 total, 8 up, 8 in 2024-03-24T12:18:35.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:35 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:35.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:35.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:35 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-67"}]: dispatch 2024-03-24T12:18:35.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-67"}]: dispatch 2024-03-24T12:18:35.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:18:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:18:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:18:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:35 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:35 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-67"}]: dispatch 2024-03-24T12:18:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-67"}]: dispatch 2024-03-24T12:18:35.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:35 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:35.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:35.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:35 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-67"}]: dispatch 2024-03-24T12:18:35.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-67"}]: dispatch 2024-03-24T12:18:36.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:36 smithi090 ceph-mon[33646]: pgmap v603: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:18:36.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:36 smithi090 ceph-mon[33646]: osdmap e416: 8 total, 8 up, 8 in 2024-03-24T12:18:36.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:36 smithi006 ceph-mon[30108]: pgmap v603: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:18:36.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:36 smithi006 ceph-mon[30108]: osdmap e416: 8 total, 8 up, 8 in 2024-03-24T12:18:36.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:36 smithi006 ceph-mon[41734]: pgmap v603: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:18:36.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:36 smithi006 ceph-mon[41734]: osdmap e416: 8 total, 8 up, 8 in 2024-03-24T12:18:37.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:37 smithi090 ceph-mon[33646]: osdmap e417: 8 total, 8 up, 8 in 2024-03-24T12:18:37.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:37.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:37 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:37.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:37 smithi006 ceph-mon[30108]: osdmap e417: 8 total, 8 up, 8 in 2024-03-24T12:18:37.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:37.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:37 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:37.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:37 smithi006 ceph-mon[41734]: osdmap e417: 8 total, 8 up, 8 in 2024-03-24T12:18:37.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:37.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:37 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:38.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:38 smithi090 ceph-mon[33646]: pgmap v606: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:18:38.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:18:38.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:38 smithi090 ceph-mon[33646]: osdmap e418: 8 total, 8 up, 8 in 2024-03-24T12:18:38.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:38 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:18:38.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:38 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:38.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:38.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:38 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-69"}]: dispatch 2024-03-24T12:18:38.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-69"}]: dispatch 2024-03-24T12:18:38.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:38 smithi006 ceph-mon[30108]: pgmap v606: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:18:38.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:18:38.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:38 smithi006 ceph-mon[30108]: osdmap e418: 8 total, 8 up, 8 in 2024-03-24T12:18:38.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:38 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:18:38.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:38 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:38.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:38.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:38 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-69"}]: dispatch 2024-03-24T12:18:38.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-69"}]: dispatch 2024-03-24T12:18:38.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:38 smithi006 ceph-mon[41734]: pgmap v606: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:18:38.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:18:38.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:38 smithi006 ceph-mon[41734]: osdmap e418: 8 total, 8 up, 8 in 2024-03-24T12:18:38.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:38 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:18:38.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:38 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:38.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:18:38.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:38 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-69"}]: dispatch 2024-03-24T12:18:38.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-69"}]: dispatch 2024-03-24T12:18:39.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:39 smithi090 ceph-mon[33646]: osdmap e419: 8 total, 8 up, 8 in 2024-03-24T12:18:39.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:39 smithi006 ceph-mon[30108]: osdmap e419: 8 total, 8 up, 8 in 2024-03-24T12:18:39.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:39 smithi006 ceph-mon[41734]: osdmap e419: 8 total, 8 up, 8 in 2024-03-24T12:18:40.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:40 smithi090 ceph-mon[33646]: pgmap v609: 260 pgs: 260 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:18:40.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:40 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:18:40.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:40 smithi090 ceph-mon[33646]: osdmap e420: 8 total, 8 up, 8 in 2024-03-24T12:18:40.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:40.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:40 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:40.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:40 smithi006 ceph-mon[30108]: pgmap v609: 260 pgs: 260 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:18:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:40 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:18:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:40 smithi006 ceph-mon[30108]: osdmap e420: 8 total, 8 up, 8 in 2024-03-24T12:18:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:40 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:40.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:40 smithi006 ceph-mon[41734]: pgmap v609: 260 pgs: 260 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:18:40.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:40 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:18:40.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:40 smithi006 ceph-mon[41734]: osdmap e420: 8 total, 8 up, 8 in 2024-03-24T12:18:40.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:40.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:40 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:18:41.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:41 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:18:41.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:41 smithi090 ceph-mon[33646]: osdmap e421: 8 total, 8 up, 8 in 2024-03-24T12:18:41.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:18:41.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:18:41.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:41 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:18:41.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:41 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:18:41.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:41 smithi006 ceph-mon[30108]: osdmap e421: 8 total, 8 up, 8 in 2024-03-24T12:18:41.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:18:41.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:18:41.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:41 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:18:41.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:41 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:18:41.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:41 smithi006 ceph-mon[41734]: osdmap e421: 8 total, 8 up, 8 in 2024-03-24T12:18:41.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:18:41.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:18:41.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:41 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:18:42.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:42 smithi006 ceph-mon[30108]: pgmap v612: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:18:42.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:42 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:18:42.612 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:42 smithi006 ceph-mon[30108]: osdmap e422: 8 total, 8 up, 8 in 2024-03-24T12:18:42.612 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:42 smithi006 ceph-mon[41734]: pgmap v612: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:18:42.613 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:42 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:18:42.613 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:42 smithi006 ceph-mon[41734]: osdmap e422: 8 total, 8 up, 8 in 2024-03-24T12:18:42.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:42 smithi090 ceph-mon[33646]: pgmap v612: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:18:42.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:42 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:18:42.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:42 smithi090 ceph-mon[33646]: osdmap e422: 8 total, 8 up, 8 in 2024-03-24T12:18:42.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:18:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:18:43.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:43 smithi090 ceph-mon[33646]: osdmap e423: 8 total, 8 up, 8 in 2024-03-24T12:18:43.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:43 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:18:43.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:43 smithi006 ceph-mon[30108]: osdmap e423: 8 total, 8 up, 8 in 2024-03-24T12:18:43.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:43 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:18:43.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:43 smithi006 ceph-mon[41734]: osdmap e423: 8 total, 8 up, 8 in 2024-03-24T12:18:43.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:43 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:18:44.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:44 smithi090 ceph-mon[33646]: pgmap v615: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-03-24T12:18:44.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:44 smithi006 ceph-mon[30108]: pgmap v615: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-03-24T12:18:44.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:44 smithi006 ceph-mon[41734]: pgmap v615: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-03-24T12:18:45.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:18:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:18:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:18:46.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:46 smithi090 ceph-mon[33646]: pgmap v616: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-03-24T12:18:46.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:46 smithi006 ceph-mon[30108]: pgmap v616: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-03-24T12:18:46.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:46 smithi006 ceph-mon[41734]: pgmap v616: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-03-24T12:18:48.682 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:48 smithi090 ceph-mon[33646]: pgmap v617: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-03-24T12:18:48.682 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:48 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:18:48.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:48 smithi006 ceph-mon[30108]: pgmap v617: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-03-24T12:18:48.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:48 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:18:48.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:48 smithi006 ceph-mon[41734]: pgmap v617: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-03-24T12:18:48.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:48 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:18:50.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:50 smithi090 ceph-mon[33646]: pgmap v618: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-03-24T12:18:50.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:50 smithi006 ceph-mon[30108]: pgmap v618: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-03-24T12:18:50.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:50 smithi006 ceph-mon[41734]: pgmap v618: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-03-24T12:18:52.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:52 smithi090 ceph-mon[33646]: pgmap v619: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-03-24T12:18:52.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:18:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:18:52.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:52 smithi006 ceph-mon[30108]: pgmap v619: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-03-24T12:18:52.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:52 smithi006 ceph-mon[41734]: pgmap v619: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-03-24T12:18:53.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:53 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:18:53.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:53 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:18:53.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:53 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:18:54.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:54 smithi090 ceph-mon[33646]: pgmap v620: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 377 B/s wr, 1 op/s 2024-03-24T12:18:54.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:54 smithi006 ceph-mon[30108]: pgmap v620: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 377 B/s wr, 1 op/s 2024-03-24T12:18:54.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:54 smithi006 ceph-mon[41734]: pgmap v620: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 377 B/s wr, 1 op/s 2024-03-24T12:18:55.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:18:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:18:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:18:56.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:56 smithi090 ceph-mon[33646]: pgmap v621: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-03-24T12:18:56.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:56 smithi006 ceph-mon[30108]: pgmap v621: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-03-24T12:18:56.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:56 smithi006 ceph-mon[41734]: pgmap v621: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-03-24T12:18:58.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:58 smithi090 ceph-mon[33646]: pgmap v622: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 597 B/s wr, 3 op/s 2024-03-24T12:18:58.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:58 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:18:58.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:58 smithi006 ceph-mon[30108]: pgmap v622: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 597 B/s wr, 3 op/s 2024-03-24T12:18:58.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:58 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:18:58.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:58 smithi006 ceph-mon[41734]: pgmap v622: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 597 B/s wr, 3 op/s 2024-03-24T12:18:58.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:58 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:18:59.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:59 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:18:59.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:59 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:18:59.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:18:59 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:18:59.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:59 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:18:59.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:59 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:18:59.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:18:59 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:18:59.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:59 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:18:59.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:59 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:18:59.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:18:59 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:19:00.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:00 smithi090 ceph-mon[33646]: pgmap v623: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-24T12:19:00.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:00 smithi006 ceph-mon[30108]: pgmap v623: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-24T12:19:00.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:00 smithi006 ceph-mon[41734]: pgmap v623: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-24T12:19:02.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:19:02 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:19:02.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:02 smithi090 ceph-mon[33646]: pgmap v624: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-24T12:19:02.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:02 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.f", "id": [7, 3]}]: dispatch 2024-03-24T12:19:02.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:02 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.f", "id": [7, 3]}]: dispatch 2024-03-24T12:19:02.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:02 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:02.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:02 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:02.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:02 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-71"}]: dispatch 2024-03-24T12:19:02.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:02 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-71"}]: dispatch 2024-03-24T12:19:02.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:02 smithi006 ceph-mon[41734]: pgmap v624: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-24T12:19:02.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:02 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.f", "id": [7, 3]}]: dispatch 2024-03-24T12:19:02.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:02 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.f", "id": [7, 3]}]: dispatch 2024-03-24T12:19:02.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:02 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:02.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:02 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:02.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:02 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-71"}]: dispatch 2024-03-24T12:19:02.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:02 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-71"}]: dispatch 2024-03-24T12:19:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:02 smithi006 ceph-mon[30108]: pgmap v624: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-24T12:19:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:02 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.f", "id": [7, 3]}]: dispatch 2024-03-24T12:19:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:02 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.f", "id": [7, 3]}]: dispatch 2024-03-24T12:19:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:02 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:02 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:02 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-71"}]: dispatch 2024-03-24T12:19:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:02 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-71"}]: dispatch 2024-03-24T12:19:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:03 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:19:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:03 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.f", "id": [7, 3]}]': finished 2024-03-24T12:19:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:03 smithi090 ceph-mon[33646]: osdmap e424: 8 total, 8 up, 8 in 2024-03-24T12:19:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:03 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:19:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:03 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:19:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:03 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.f", "id": [7, 3]}]': finished 2024-03-24T12:19:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:03 smithi006 ceph-mon[30108]: osdmap e424: 8 total, 8 up, 8 in 2024-03-24T12:19:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:03 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:19:03.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:03 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:19:03.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:03 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.f", "id": [7, 3]}]': finished 2024-03-24T12:19:03.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:03 smithi006 ceph-mon[41734]: osdmap e424: 8 total, 8 up, 8 in 2024-03-24T12:19:03.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:03 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:19:04.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:04 smithi090 ceph-mon[33646]: pgmap v626: 260 pgs: 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-03-24T12:19:04.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:04 smithi090 ceph-mon[33646]: osdmap e425: 8 total, 8 up, 8 in 2024-03-24T12:19:04.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:04 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:04.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:04 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:04 smithi006 ceph-mon[30108]: pgmap v626: 260 pgs: 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-03-24T12:19:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:04 smithi006 ceph-mon[30108]: osdmap e425: 8 total, 8 up, 8 in 2024-03-24T12:19:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:04 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:04 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:04 smithi006 ceph-mon[41734]: pgmap v626: 260 pgs: 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-03-24T12:19:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:04 smithi006 ceph-mon[41734]: osdmap e425: 8 total, 8 up, 8 in 2024-03-24T12:19:04.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:04 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:04.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:04 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:05.659 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:19:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:19:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:19:05.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:05 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:19:05.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:05 smithi090 ceph-mon[33646]: osdmap e426: 8 total, 8 up, 8 in 2024-03-24T12:19:05.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:05 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:19:05.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:05 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:19:05.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:05 smithi006 ceph-mon[30108]: osdmap e426: 8 total, 8 up, 8 in 2024-03-24T12:19:05.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:05 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:19:05.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:05 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:19:05.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:05 smithi006 ceph-mon[41734]: osdmap e426: 8 total, 8 up, 8 in 2024-03-24T12:19:05.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:05 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:19:06.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:06 smithi090 ceph-mon[33646]: pgmap v629: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:19:06.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:06 smithi090 ceph-mon[33646]: osdmap e427: 8 total, 8 up, 8 in 2024-03-24T12:19:06.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:06 smithi006 ceph-mon[30108]: pgmap v629: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:19:06.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:06 smithi006 ceph-mon[30108]: osdmap e427: 8 total, 8 up, 8 in 2024-03-24T12:19:06.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:06 smithi006 ceph-mon[41734]: pgmap v629: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:19:06.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:06 smithi006 ceph-mon[41734]: osdmap e427: 8 total, 8 up, 8 in 2024-03-24T12:19:07.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:07 smithi090 ceph-mon[33646]: osdmap e428: 8 total, 8 up, 8 in 2024-03-24T12:19:07.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:07 smithi006 ceph-mon[30108]: osdmap e428: 8 total, 8 up, 8 in 2024-03-24T12:19:07.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:07 smithi006 ceph-mon[41734]: osdmap e428: 8 total, 8 up, 8 in 2024-03-24T12:19:08.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:08 smithi090 ceph-mon[33646]: pgmap v632: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-24T12:19:08.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:08 smithi006 ceph-mon[30108]: pgmap v632: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-24T12:19:08.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:08 smithi006 ceph-mon[41734]: pgmap v632: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-24T12:19:09.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:09 smithi006 ceph-mon[30108]: pgmap v633: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-03-24T12:19:09.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:09 smithi006 ceph-mon[41734]: pgmap v633: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-03-24T12:19:10.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:09 smithi090 ceph-mon[33646]: pgmap v633: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-03-24T12:19:12.516 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:12 smithi090 ceph-mon[33646]: pgmap v634: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-03-24T12:19:12.664 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:12 smithi006 ceph-mon[30108]: pgmap v634: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-03-24T12:19:12.664 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:12 smithi006 ceph-mon[41734]: pgmap v634: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-03-24T12:19:12.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:19:12 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:19:13.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:13 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:19:13.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:13 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:19:13.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:13 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:19:14.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:14 smithi090 ceph-mon[33646]: pgmap v635: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-03-24T12:19:14.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:14 smithi006 ceph-mon[30108]: pgmap v635: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-03-24T12:19:14.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:14 smithi006 ceph-mon[41734]: pgmap v635: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-03-24T12:19:15.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:19:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:19:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:19:16.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:16 smithi090 ceph-mon[33646]: pgmap v636: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 752 B/s wr, 3 op/s 2024-03-24T12:19:16.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:16 smithi006 ceph-mon[30108]: pgmap v636: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 752 B/s wr, 3 op/s 2024-03-24T12:19:16.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:16 smithi006 ceph-mon[41734]: pgmap v636: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 752 B/s wr, 3 op/s 2024-03-24T12:19:17.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:17.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:17.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-73"}]: dispatch 2024-03-24T12:19:17.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-73"}]: dispatch 2024-03-24T12:19:17.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:17.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:17.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-73"}]: dispatch 2024-03-24T12:19:17.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-73"}]: dispatch 2024-03-24T12:19:17.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:17.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:17.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-73"}]: dispatch 2024-03-24T12:19:17.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-73"}]: dispatch 2024-03-24T12:19:18.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:18 smithi090 ceph-mon[33646]: pgmap v637: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 682 B/s wr, 3 op/s 2024-03-24T12:19:18.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:18 smithi090 ceph-mon[33646]: osdmap e429: 8 total, 8 up, 8 in 2024-03-24T12:19:18.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:18 smithi006 ceph-mon[30108]: pgmap v637: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 682 B/s wr, 3 op/s 2024-03-24T12:19:18.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:18 smithi006 ceph-mon[30108]: osdmap e429: 8 total, 8 up, 8 in 2024-03-24T12:19:18.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:18 smithi006 ceph-mon[41734]: pgmap v637: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 682 B/s wr, 3 op/s 2024-03-24T12:19:18.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:18 smithi006 ceph-mon[41734]: osdmap e429: 8 total, 8 up, 8 in 2024-03-24T12:19:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:19 smithi090 ceph-mon[33646]: osdmap e430: 8 total, 8 up, 8 in 2024-03-24T12:19:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:19:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:19:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:19 smithi090 ceph-mon[33646]: osdmap e431: 8 total, 8 up, 8 in 2024-03-24T12:19:19.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:19 smithi006 ceph-mon[30108]: osdmap e430: 8 total, 8 up, 8 in 2024-03-24T12:19:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:19:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:19:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:19 smithi006 ceph-mon[30108]: osdmap e431: 8 total, 8 up, 8 in 2024-03-24T12:19:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:19 smithi006 ceph-mon[41734]: osdmap e430: 8 total, 8 up, 8 in 2024-03-24T12:19:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:19:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:19:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:19 smithi006 ceph-mon[41734]: osdmap e431: 8 total, 8 up, 8 in 2024-03-24T12:19:20.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:20 smithi090 ceph-mon[33646]: pgmap v640: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T12:19:20.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:20 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:19:20.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:20 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:19:20.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:20 smithi090 ceph-mon[33646]: osdmap e432: 8 total, 8 up, 8 in 2024-03-24T12:19:20.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:20 smithi006 ceph-mon[30108]: pgmap v640: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T12:19:20.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:20 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:19:20.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:20 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:19:20.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:20 smithi006 ceph-mon[30108]: osdmap e432: 8 total, 8 up, 8 in 2024-03-24T12:19:20.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:20 smithi006 ceph-mon[41734]: pgmap v640: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T12:19:20.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:20 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:19:20.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:20 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:19:20.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:20 smithi006 ceph-mon[41734]: osdmap e432: 8 total, 8 up, 8 in 2024-03-24T12:19:22.522 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:22 smithi090 ceph-mon[33646]: pgmap v643: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:19:22.522 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:22 smithi090 ceph-mon[33646]: osdmap e433: 8 total, 8 up, 8 in 2024-03-24T12:19:22.522 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:22.522 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:22.523 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-75"}]: dispatch 2024-03-24T12:19:22.523 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-75"}]: dispatch 2024-03-24T12:19:22.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:22 smithi006 ceph-mon[30108]: pgmap v643: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:19:22.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:22 smithi006 ceph-mon[30108]: osdmap e433: 8 total, 8 up, 8 in 2024-03-24T12:19:22.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:22.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:22.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-75"}]: dispatch 2024-03-24T12:19:22.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-75"}]: dispatch 2024-03-24T12:19:22.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:22 smithi006 ceph-mon[41734]: pgmap v643: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:19:22.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:22 smithi006 ceph-mon[41734]: osdmap e433: 8 total, 8 up, 8 in 2024-03-24T12:19:22.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:22.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:22.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-75"}]: dispatch 2024-03-24T12:19:22.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-75"}]: dispatch 2024-03-24T12:19:22.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:19:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:19:23.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:23 smithi090 ceph-mon[33646]: osdmap e434: 8 total, 8 up, 8 in 2024-03-24T12:19:23.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:23 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:19:23.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:23 smithi006 ceph-mon[30108]: osdmap e434: 8 total, 8 up, 8 in 2024-03-24T12:19:23.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:23 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:19:23.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:23 smithi006 ceph-mon[41734]: osdmap e434: 8 total, 8 up, 8 in 2024-03-24T12:19:23.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:23 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:19:24.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:24 smithi090 ceph-mon[33646]: pgmap v646: 260 pgs: 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-24T12:19:24.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:24 smithi090 ceph-mon[33646]: osdmap e435: 8 total, 8 up, 8 in 2024-03-24T12:19:24.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:24.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:24.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:24 smithi006 ceph-mon[30108]: pgmap v646: 260 pgs: 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-24T12:19:24.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:24 smithi006 ceph-mon[30108]: osdmap e435: 8 total, 8 up, 8 in 2024-03-24T12:19:24.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:24.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:24.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:24 smithi006 ceph-mon[41734]: pgmap v646: 260 pgs: 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-24T12:19:24.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:24 smithi006 ceph-mon[41734]: osdmap e435: 8 total, 8 up, 8 in 2024-03-24T12:19:24.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:24.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:25.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:25 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:19:25.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:25 smithi090 ceph-mon[33646]: osdmap e436: 8 total, 8 up, 8 in 2024-03-24T12:19:25.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:19:25.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:25 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:19:25.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:25 smithi006 ceph-mon[30108]: osdmap e436: 8 total, 8 up, 8 in 2024-03-24T12:19:25.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:19:25.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:19:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:19:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:19:25.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:25 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:19:25.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:25 smithi006 ceph-mon[41734]: osdmap e436: 8 total, 8 up, 8 in 2024-03-24T12:19:25.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:19:26.579 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:26 smithi090 ceph-mon[33646]: pgmap v649: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-24T12:19:26.579 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:26 smithi090 ceph-mon[33646]: osdmap e437: 8 total, 8 up, 8 in 2024-03-24T12:19:26.579 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:26.579 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:26.580 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-77"}]: dispatch 2024-03-24T12:19:26.580 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-77"}]: dispatch 2024-03-24T12:19:26.580 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:26 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:19:26.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:26 smithi006 ceph-mon[30108]: pgmap v649: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-24T12:19:26.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:26 smithi006 ceph-mon[30108]: osdmap e437: 8 total, 8 up, 8 in 2024-03-24T12:19:26.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:26.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:26.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-77"}]: dispatch 2024-03-24T12:19:26.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-77"}]: dispatch 2024-03-24T12:19:26.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:26 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:19:26.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:26 smithi006 ceph-mon[41734]: pgmap v649: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-24T12:19:26.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:26 smithi006 ceph-mon[41734]: osdmap e437: 8 total, 8 up, 8 in 2024-03-24T12:19:26.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:26.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:26.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-77"}]: dispatch 2024-03-24T12:19:26.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-77"}]: dispatch 2024-03-24T12:19:26.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:26 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:19:27.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:27 smithi090 ceph-mon[33646]: osdmap e438: 8 total, 8 up, 8 in 2024-03-24T12:19:27.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:27 smithi006 ceph-mon[30108]: osdmap e438: 8 total, 8 up, 8 in 2024-03-24T12:19:27.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:27 smithi006 ceph-mon[41734]: osdmap e438: 8 total, 8 up, 8 in 2024-03-24T12:19:28.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:28 smithi090 ceph-mon[33646]: pgmap v652: 260 pgs: 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:19:28.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:28 smithi090 ceph-mon[33646]: osdmap e439: 8 total, 8 up, 8 in 2024-03-24T12:19:28.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:28.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:28 smithi006 ceph-mon[30108]: pgmap v652: 260 pgs: 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:19:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:28 smithi006 ceph-mon[30108]: osdmap e439: 8 total, 8 up, 8 in 2024-03-24T12:19:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:28.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:28 smithi006 ceph-mon[41734]: pgmap v652: 260 pgs: 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:19:28.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:28 smithi006 ceph-mon[41734]: osdmap e439: 8 total, 8 up, 8 in 2024-03-24T12:19:28.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:28.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:29.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:19:29.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:29 smithi090 ceph-mon[33646]: osdmap e440: 8 total, 8 up, 8 in 2024-03-24T12:19:29.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:29 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:19:29.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:19:29.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:29 smithi006 ceph-mon[30108]: osdmap e440: 8 total, 8 up, 8 in 2024-03-24T12:19:29.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:29 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:19:29.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:19:29.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:29 smithi006 ceph-mon[41734]: osdmap e440: 8 total, 8 up, 8 in 2024-03-24T12:19:29.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:29 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:19:30.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:30 smithi090 ceph-mon[33646]: pgmap v655: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:19:30.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:30 smithi090 ceph-mon[33646]: osdmap e441: 8 total, 8 up, 8 in 2024-03-24T12:19:30.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:30 smithi006 ceph-mon[30108]: pgmap v655: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:19:30.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:30 smithi006 ceph-mon[30108]: osdmap e441: 8 total, 8 up, 8 in 2024-03-24T12:19:30.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:30 smithi006 ceph-mon[41734]: pgmap v655: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:19:30.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:30 smithi006 ceph-mon[41734]: osdmap e441: 8 total, 8 up, 8 in 2024-03-24T12:19:31.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:31 smithi090 ceph-mon[33646]: osdmap e442: 8 total, 8 up, 8 in 2024-03-24T12:19:31.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.18"}]: dispatch 2024-03-24T12:19:31.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:31 smithi090 ceph-mon[33646]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.18"}]: dispatch 2024-03-24T12:19:31.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:31 smithi090 ceph-mon[33646]: 304.18 deep-scrub starts 2024-03-24T12:19:31.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:31 smithi090 ceph-mon[33646]: 304.18 deep-scrub ok 2024-03-24T12:19:31.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:31 smithi006 ceph-mon[30108]: osdmap e442: 8 total, 8 up, 8 in 2024-03-24T12:19:31.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.18"}]: dispatch 2024-03-24T12:19:31.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:31 smithi006 ceph-mon[30108]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.18"}]: dispatch 2024-03-24T12:19:31.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:31 smithi006 ceph-mon[30108]: 304.18 deep-scrub starts 2024-03-24T12:19:31.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:31 smithi006 ceph-mon[30108]: 304.18 deep-scrub ok 2024-03-24T12:19:31.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:31 smithi006 ceph-mon[41734]: osdmap e442: 8 total, 8 up, 8 in 2024-03-24T12:19:31.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.18"}]: dispatch 2024-03-24T12:19:31.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:31 smithi006 ceph-mon[41734]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.18"}]: dispatch 2024-03-24T12:19:31.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:31 smithi006 ceph-mon[41734]: 304.18 deep-scrub starts 2024-03-24T12:19:31.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:31 smithi006 ceph-mon[41734]: 304.18 deep-scrub ok 2024-03-24T12:19:32.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:32 smithi090 ceph-mon[33646]: pgmap v658: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:19:32.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:19:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:19:32.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:32 smithi006 ceph-mon[30108]: pgmap v658: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:19:32.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:32 smithi006 ceph-mon[41734]: pgmap v658: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:19:33.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:33 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:19:33.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:33 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:19:33.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:33 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:19:33.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:33 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:19:33.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:33 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:19:33.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:33 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:19:34.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:34 smithi090 ceph-mon[33646]: pgmap v659: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s rd, 874 B/s wr, 124 op/s 2024-03-24T12:19:34.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:34 smithi006 ceph-mon[30108]: pgmap v659: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s rd, 874 B/s wr, 124 op/s 2024-03-24T12:19:34.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:34 smithi006 ceph-mon[41734]: pgmap v659: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s rd, 874 B/s wr, 124 op/s 2024-03-24T12:19:35.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:19:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:19:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:19:36.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:36 smithi090 ceph-mon[33646]: pgmap v660: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s rd, 747 B/s wr, 106 op/s 2024-03-24T12:19:36.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:36 smithi006 ceph-mon[30108]: pgmap v660: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s rd, 747 B/s wr, 106 op/s 2024-03-24T12:19:36.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:36 smithi006 ceph-mon[41734]: pgmap v660: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s rd, 747 B/s wr, 106 op/s 2024-03-24T12:19:38.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:38 smithi090 ceph-mon[33646]: pgmap v661: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 639 B/s wr, 91 op/s 2024-03-24T12:19:38.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:38 smithi006 ceph-mon[30108]: pgmap v661: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 639 B/s wr, 91 op/s 2024-03-24T12:19:38.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:38 smithi006 ceph-mon[41734]: pgmap v661: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 639 B/s wr, 91 op/s 2024-03-24T12:19:40.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:40 smithi090 ceph-mon[33646]: pgmap v662: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 45 KiB/s rd, 520 B/s wr, 74 op/s 2024-03-24T12:19:40.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:40 smithi006 ceph-mon[30108]: pgmap v662: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 45 KiB/s rd, 520 B/s wr, 74 op/s 2024-03-24T12:19:40.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:40 smithi006 ceph-mon[41734]: pgmap v662: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 45 KiB/s rd, 520 B/s wr, 74 op/s 2024-03-24T12:19:42.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:42 smithi090 ceph-mon[33646]: pgmap v663: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 40 KiB/s rd, 472 B/s wr, 67 op/s 2024-03-24T12:19:42.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:19:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:19:42.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:42 smithi006 ceph-mon[30108]: pgmap v663: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 40 KiB/s rd, 472 B/s wr, 67 op/s 2024-03-24T12:19:42.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:42 smithi006 ceph-mon[41734]: pgmap v663: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 40 KiB/s rd, 472 B/s wr, 67 op/s 2024-03-24T12:19:43.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:43 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:19:43.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:43 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:19:43.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:43 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:19:44.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:44 smithi090 ceph-mon[33646]: pgmap v664: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 426 B/s wr, 61 op/s 2024-03-24T12:19:44.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:44 smithi006 ceph-mon[30108]: pgmap v664: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 426 B/s wr, 61 op/s 2024-03-24T12:19:44.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:44 smithi006 ceph-mon[41734]: pgmap v664: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 426 B/s wr, 61 op/s 2024-03-24T12:19:45.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:19:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:19:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:19:46.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:46 smithi090 ceph-mon[33646]: pgmap v665: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:19:46.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:46 smithi006 ceph-mon[30108]: pgmap v665: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:19:46.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:46 smithi006 ceph-mon[41734]: pgmap v665: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:19:48.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:48 smithi090 ceph-mon[33646]: pgmap v666: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:19:48.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:48 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:19:48.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:48 smithi006 ceph-mon[30108]: pgmap v666: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:19:48.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:48 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:19:48.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:48 smithi006 ceph-mon[41734]: pgmap v666: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:19:48.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:48 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:19:50.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:50 smithi090 ceph-mon[33646]: pgmap v667: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:19:50.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:50 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:50.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:50 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:50.679 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:19:50 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:19:50.393+0000 7fdae71d2700 -1 snap_mapper.add_oid found existing snaps mapped on 304:1e36b853:test-rados-api-smithi006-82061-80::foo:2, removing 2024-03-24T12:19:50.679 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:19:50 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:19:50.393+0000 7fb21b2ed700 -1 snap_mapper.add_oid found existing snaps mapped on 304:1e36b853:test-rados-api-smithi006-82061-80::foo:2, removing 2024-03-24T12:19:50.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:50 smithi006 ceph-mon[30108]: pgmap v667: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:19:50.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:50 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:50.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:50 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:50.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:50 smithi006 ceph-mon[41734]: pgmap v667: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:19:50.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:50 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:50.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:50 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:50.712 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:19:50 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-2[58188]: 2024-03-24T12:19:50.394+0000 7f5f86f3d700 -1 snap_mapper.add_oid found existing snaps mapped on 304:1e36b853:test-rados-api-smithi006-82061-80::foo:2, removing 2024-03-24T12:19:51.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:51 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-79"}]: dispatch 2024-03-24T12:19:51.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:51 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-79"}]: dispatch 2024-03-24T12:19:51.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-79"}]: dispatch 2024-03-24T12:19:51.716 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:51 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-79"}]: dispatch 2024-03-24T12:19:51.716 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:51 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-79"}]: dispatch 2024-03-24T12:19:51.716 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:51 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-79"}]: dispatch 2024-03-24T12:19:52.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:52 smithi090 ceph-mon[33646]: pgmap v668: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:19:52.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:52 smithi090 ceph-mon[33646]: osdmap e443: 8 total, 8 up, 8 in 2024-03-24T12:19:52.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:19:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:19:52.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:52 smithi006 ceph-mon[30108]: pgmap v668: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:19:52.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:52 smithi006 ceph-mon[30108]: osdmap e443: 8 total, 8 up, 8 in 2024-03-24T12:19:52.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:52 smithi006 ceph-mon[41734]: pgmap v668: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:19:52.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:52 smithi006 ceph-mon[41734]: osdmap e443: 8 total, 8 up, 8 in 2024-03-24T12:19:53.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:53 smithi090 ceph-mon[33646]: osdmap e444: 8 total, 8 up, 8 in 2024-03-24T12:19:53.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:53 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:53.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:53 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:53.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:53 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:19:53.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:53 smithi006 ceph-mon[30108]: osdmap e444: 8 total, 8 up, 8 in 2024-03-24T12:19:53.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:53 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:53.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:53 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:53.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:53 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:19:53.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:53 smithi006 ceph-mon[41734]: osdmap e444: 8 total, 8 up, 8 in 2024-03-24T12:19:53.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:53 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:53.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:53 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:19:53.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:53 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:19:54.542 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc0fe0 2024-03-24T12:19:54.542 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc1110 2024-03-24T12:19:54.542 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc17f0 2024-03-24T12:19:54.542 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc1350 2024-03-24T12:19:54.543 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc1480 2024-03-24T12:19:54.543 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc1590 2024-03-24T12:19:54.543 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc16e0 2024-03-24T12:19:54.543 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc1dc0 2024-03-24T12:19:54.543 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc1920 2024-03-24T12:19:54.543 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc1a50 2024-03-24T12:19:54.543 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc1b60 2024-03-24T12:19:54.543 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc1cb0 2024-03-24T12:19:54.543 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc23b0 2024-03-24T12:19:54.543 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc1f10 2024-03-24T12:19:54.543 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc2060 2024-03-24T12:19:54.543 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc2190 2024-03-24T12:19:54.543 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc2500 2024-03-24T12:19:54.543 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc29e0 2024-03-24T12:19:54.544 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc2b60 2024-03-24T12:19:54.544 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc2630 2024-03-24T12:19:54.544 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc2760 2024-03-24T12:19:54.544 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc2890 2024-03-24T12:19:54.544 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc2c90 2024-03-24T12:19:54.544 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc3120 2024-03-24T12:19:54.544 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc3270 2024-03-24T12:19:54.544 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbfc40 2024-03-24T12:19:54.544 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbfd70 2024-03-24T12:19:54.544 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc3840 2024-03-24T12:19:54.544 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc33a0 2024-03-24T12:19:54.544 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc34d0 2024-03-24T12:19:54.544 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc3600 2024-03-24T12:19:54.544 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc3730 2024-03-24T12:19:54.544 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc3e10 2024-03-24T12:19:54.545 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc3970 2024-03-24T12:19:54.545 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc3ac0 2024-03-24T12:19:54.545 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc3bf0 2024-03-24T12:19:54.545 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc3f40 2024-03-24T12:19:54.545 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc43d0 2024-03-24T12:19:54.545 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc4500 2024-03-24T12:19:54.545 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbfe80 2024-03-24T12:19:54.545 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc4680 2024-03-24T12:19:54.545 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc4820 2024-03-24T12:19:54.545 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adbd8b0 2024-03-24T12:19:54.545 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc2da0 2024-03-24T12:19:54.545 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc2ed0 2024-03-24T12:19:54.545 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc4970 2024-03-24T12:19:54.545 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc4e00 2024-03-24T12:19:54.545 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc4f30 2024-03-24T12:19:54.546 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc5500 2024-03-24T12:19:54.546 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc4aa0 2024-03-24T12:19:54.546 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc4bd0 2024-03-24T12:19:54.546 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc5060 2024-03-24T12:19:54.546 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc5190 2024-03-24T12:19:54.546 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc4ce0 2024-03-24T12:19:54.546 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc52c0 2024-03-24T12:19:54.546 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc53f0 2024-03-24T12:19:54.546 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc5650 2024-03-24T12:19:54.546 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc5ac0 2024-03-24T12:19:54.546 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc5c50 2024-03-24T12:19:54.546 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc4050 2024-03-24T12:19:54.546 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc4180 2024-03-24T12:19:54.546 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc6240 2024-03-24T12:19:54.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc5da0 2024-03-24T12:19:54.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc5ed0 2024-03-24T12:19:54.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc6000 2024-03-24T12:19:54.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc6130 2024-03-24T12:19:54.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc6810 2024-03-24T12:19:54.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc6370 2024-03-24T12:19:54.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc64c0 2024-03-24T12:19:54.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc65f0 2024-03-24T12:19:54.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc6940 2024-03-24T12:19:54.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc6dd0 2024-03-24T12:19:54.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc6f00 2024-03-24T12:19:54.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc5760 2024-03-24T12:19:54.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc5890 2024-03-24T12:19:54.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc74d0 2024-03-24T12:19:54.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc7030 2024-03-24T12:19:54.547 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc7160 2024-03-24T12:19:54.548 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc7290 2024-03-24T12:19:54.548 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc73c0 2024-03-24T12:19:54.548 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc7aa0 2024-03-24T12:19:54.548 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc7600 2024-03-24T12:19:54.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:54 smithi006 ceph-mon[30108]: pgmap v671: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-03-24T12:19:54.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:54 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:19:54.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:54 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:19:54.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:54 smithi006 ceph-mon[30108]: osdmap e445: 8 total, 8 up, 8 in 2024-03-24T12:19:54.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:19:54.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:19:54.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:54 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:19:54.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:54 smithi006 ceph-mon[41734]: pgmap v671: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-03-24T12:19:54.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:54 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:19:54.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:54 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:19:54.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:54 smithi006 ceph-mon[41734]: osdmap e445: 8 total, 8 up, 8 in 2024-03-24T12:19:54.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:19:54.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:19:54.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:54 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:19:54.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:54 smithi090 ceph-mon[33646]: pgmap v671: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-03-24T12:19:54.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:54 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:19:54.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:54 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:19:54.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:54 smithi090 ceph-mon[33646]: osdmap e445: 8 total, 8 up, 8 in 2024-03-24T12:19:54.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:19:54.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:19:54.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:54 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:19:55.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:19:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:19:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:19:55.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:55 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:19:55.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:55 smithi006 ceph-mon[30108]: osdmap e446: 8 total, 8 up, 8 in 2024-03-24T12:19:55.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:55 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:55.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:55 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:55.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:55 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:19:55.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:55 smithi006 ceph-mon[41734]: osdmap e446: 8 total, 8 up, 8 in 2024-03-24T12:19:55.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:55 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:55.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:55 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:55.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:55 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:19:55.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:55 smithi090 ceph-mon[33646]: osdmap e446: 8 total, 8 up, 8 in 2024-03-24T12:19:55.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:55 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:55.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:55 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:19:56.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:56 smithi006 ceph-mon[30108]: pgmap v674: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-03-24T12:19:56.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:56 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:19:56.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:56 smithi006 ceph-mon[30108]: osdmap e447: 8 total, 8 up, 8 in 2024-03-24T12:19:56.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:56 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:19:56.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:56 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:19:56.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:56 smithi006 ceph-mon[41734]: pgmap v674: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-03-24T12:19:56.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:56 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:19:56.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:56 smithi006 ceph-mon[41734]: osdmap e447: 8 total, 8 up, 8 in 2024-03-24T12:19:56.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:56 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:19:56.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:56 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:19:56.828 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:56 smithi090 ceph-mon[33646]: pgmap v674: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-03-24T12:19:56.828 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:56 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:19:56.828 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:56 smithi090 ceph-mon[33646]: osdmap e447: 8 total, 8 up, 8 in 2024-03-24T12:19:56.828 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:56 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:19:56.829 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:56 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:19:57.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:57 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-24T12:19:57.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:57 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:19:57.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:57 smithi006 ceph-mon[30108]: osdmap e448: 8 total, 8 up, 8 in 2024-03-24T12:19:57.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:57 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:19:57.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:57 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-24T12:19:57.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:57 smithi006 ceph-mon[30108]: osdmap e449: 8 total, 8 up, 8 in 2024-03-24T12:19:57.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:57 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-24T12:19:57.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:57 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:19:57.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:57 smithi006 ceph-mon[41734]: osdmap e448: 8 total, 8 up, 8 in 2024-03-24T12:19:57.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:57 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:19:57.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:57 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-24T12:19:57.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:57 smithi006 ceph-mon[41734]: osdmap e449: 8 total, 8 up, 8 in 2024-03-24T12:19:57.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:57 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-24T12:19:57.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:57 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:19:57.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:57 smithi090 ceph-mon[33646]: osdmap e448: 8 total, 8 up, 8 in 2024-03-24T12:19:57.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:57 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:19:57.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:57 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-24T12:19:57.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:57 smithi090 ceph-mon[33646]: osdmap e449: 8 total, 8 up, 8 in 2024-03-24T12:19:58.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:58 smithi006 ceph-mon[30108]: pgmap v677: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:19:58.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:58 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-24T12:19:58.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:58 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-24T12:19:58.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:58 smithi006 ceph-mon[41734]: pgmap v677: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:19:58.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:58 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-24T12:19:58.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:58 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-24T12:19:58.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:58 smithi090 ceph-mon[33646]: pgmap v677: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:19:58.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:58 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-24T12:19:58.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:58 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-24T12:19:59.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:59 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-03-24T12:19:59.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:59 smithi090 ceph-mon[33646]: osdmap e450: 8 total, 8 up, 8 in 2024-03-24T12:19:59.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:59 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-24T12:19:59.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:59 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-24T12:19:59.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:59 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:19:59.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:59 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:19:59.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:59 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:19:59.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:19:59 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:19:59.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:59 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-03-24T12:19:59.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:59 smithi006 ceph-mon[30108]: osdmap e450: 8 total, 8 up, 8 in 2024-03-24T12:19:59.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:59 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-24T12:19:59.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:59 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-24T12:19:59.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:59 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:19:59.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:59 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:19:59.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:59 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:19:59.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:19:59 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:19:59.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:59 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-03-24T12:19:59.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:59 smithi006 ceph-mon[41734]: osdmap e450: 8 total, 8 up, 8 in 2024-03-24T12:19:59.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:59 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-24T12:19:59.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:59 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-24T12:19:59.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:59 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:19:59.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:59 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:19:59.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:59 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:19:59.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:19:59 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:20:00.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:00 smithi090 ceph-mon[33646]: pgmap v680: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:20:00.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:00 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-03-24T12:20:00.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:00 smithi090 ceph-mon[33646]: osdmap e451: 8 total, 8 up, 8 in 2024-03-24T12:20:00.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:00 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:00.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:00 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:00.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:00 smithi090 ceph-mon[33646]: Health detail: HEALTH_WARN 3 pool(s) do not have an application enabled 2024-03-24T12:20:00.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:00 smithi090 ceph-mon[33646]: [WRN] POOL_APP_NOT_ENABLED: 3 pool(s) do not have an application enabled 2024-03-24T12:20:00.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:00 smithi090 ceph-mon[33646]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-24T12:20:00.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:00 smithi090 ceph-mon[33646]: application not enabled on pool 'poolrace.1804289383' 2024-03-24T12:20:00.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:00 smithi090 ceph-mon[33646]: application not enabled on pool 'poolrace2.846930886' 2024-03-24T12:20:00.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:00 smithi090 ceph-mon[33646]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-24T12:20:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:00 smithi006 ceph-mon[30108]: pgmap v680: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:20:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:00 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-03-24T12:20:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:00 smithi006 ceph-mon[30108]: osdmap e451: 8 total, 8 up, 8 in 2024-03-24T12:20:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:00 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:00 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:00 smithi006 ceph-mon[30108]: Health detail: HEALTH_WARN 3 pool(s) do not have an application enabled 2024-03-24T12:20:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:00 smithi006 ceph-mon[30108]: [WRN] POOL_APP_NOT_ENABLED: 3 pool(s) do not have an application enabled 2024-03-24T12:20:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:00 smithi006 ceph-mon[30108]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-24T12:20:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:00 smithi006 ceph-mon[30108]: application not enabled on pool 'poolrace.1804289383' 2024-03-24T12:20:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:00 smithi006 ceph-mon[30108]: application not enabled on pool 'poolrace2.846930886' 2024-03-24T12:20:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:00 smithi006 ceph-mon[30108]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-24T12:20:00.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:00 smithi006 ceph-mon[41734]: pgmap v680: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:20:00.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:00 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-03-24T12:20:00.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:00 smithi006 ceph-mon[41734]: osdmap e451: 8 total, 8 up, 8 in 2024-03-24T12:20:00.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:00 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:00.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:00 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:00.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:00 smithi006 ceph-mon[41734]: Health detail: HEALTH_WARN 3 pool(s) do not have an application enabled 2024-03-24T12:20:00.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:00 smithi006 ceph-mon[41734]: [WRN] POOL_APP_NOT_ENABLED: 3 pool(s) do not have an application enabled 2024-03-24T12:20:00.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:00 smithi006 ceph-mon[41734]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-24T12:20:00.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:00 smithi006 ceph-mon[41734]: application not enabled on pool 'poolrace.1804289383' 2024-03-24T12:20:00.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:00 smithi006 ceph-mon[41734]: application not enabled on pool 'poolrace2.846930886' 2024-03-24T12:20:00.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:00 smithi006 ceph-mon[41734]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-24T12:20:01.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-24T12:20:01.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:01 smithi090 ceph-mon[33646]: osdmap e452: 8 total, 8 up, 8 in 2024-03-24T12:20:01.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:01.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:01.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-81"}]: dispatch 2024-03-24T12:20:01.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-81"}]: dispatch 2024-03-24T12:20:01.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:01 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:20:01.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-24T12:20:01.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:01 smithi006 ceph-mon[41734]: osdmap e452: 8 total, 8 up, 8 in 2024-03-24T12:20:01.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:01.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:01.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-81"}]: dispatch 2024-03-24T12:20:01.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-81"}]: dispatch 2024-03-24T12:20:01.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:01 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:20:01.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-24T12:20:01.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:01 smithi006 ceph-mon[30108]: osdmap e452: 8 total, 8 up, 8 in 2024-03-24T12:20:01.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:01.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:01.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-81"}]: dispatch 2024-03-24T12:20:01.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-81"}]: dispatch 2024-03-24T12:20:01.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:01 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:20:02.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:02 smithi090 ceph-mon[33646]: pgmap v683: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:20:02.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:02 smithi090 ceph-mon[33646]: osdmap e453: 8 total, 8 up, 8 in 2024-03-24T12:20:02.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:20:02 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:20:02.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:02 smithi006 ceph-mon[30108]: pgmap v683: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:20:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:02 smithi006 ceph-mon[30108]: osdmap e453: 8 total, 8 up, 8 in 2024-03-24T12:20:02.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:02 smithi006 ceph-mon[41734]: pgmap v683: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:20:02.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:02 smithi006 ceph-mon[41734]: osdmap e453: 8 total, 8 up, 8 in 2024-03-24T12:20:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:03 smithi090 ceph-mon[33646]: osdmap e454: 8 total, 8 up, 8 in 2024-03-24T12:20:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:03 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:03 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:20:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:03 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:20:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:03 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:20:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:03 smithi090 ceph-mon[33646]: osdmap e455: 8 total, 8 up, 8 in 2024-03-24T12:20:03.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:20:03.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:03 smithi006 ceph-mon[30108]: osdmap e454: 8 total, 8 up, 8 in 2024-03-24T12:20:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:03 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:03 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:20:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:03 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:20:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:03 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:20:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:03 smithi006 ceph-mon[30108]: osdmap e455: 8 total, 8 up, 8 in 2024-03-24T12:20:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:20:03.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:03 smithi006 ceph-mon[41734]: osdmap e454: 8 total, 8 up, 8 in 2024-03-24T12:20:03.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:03.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:03 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:03.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:03 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:20:03.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:03 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:20:03.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:03 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:20:03.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:03 smithi006 ceph-mon[41734]: osdmap e455: 8 total, 8 up, 8 in 2024-03-24T12:20:03.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:20:04.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:04 smithi090 ceph-mon[33646]: pgmap v686: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-03-24T12:20:04.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:04 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:04.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:04 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:04.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:04 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:20:04.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:04 smithi090 ceph-mon[33646]: osdmap e456: 8 total, 8 up, 8 in 2024-03-24T12:20:04.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:04 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:04.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:04 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:04 smithi006 ceph-mon[30108]: pgmap v686: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-03-24T12:20:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:04 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:04 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:04 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:20:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:04 smithi006 ceph-mon[30108]: osdmap e456: 8 total, 8 up, 8 in 2024-03-24T12:20:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:04 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:04 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:04 smithi006 ceph-mon[41734]: pgmap v686: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-03-24T12:20:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:04 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:04 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:04 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:20:04.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:04 smithi006 ceph-mon[41734]: osdmap e456: 8 total, 8 up, 8 in 2024-03-24T12:20:04.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:04 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:04.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:04 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:05.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:20:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:20:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:20:06.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:06 smithi090 ceph-mon[33646]: pgmap v689: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-03-24T12:20:06.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:06 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:20:06.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:06 smithi090 ceph-mon[33646]: osdmap e457: 8 total, 8 up, 8 in 2024-03-24T12:20:06.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:06 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:20:06.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:06 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:20:06.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:06 smithi006 ceph-mon[30108]: pgmap v689: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-03-24T12:20:06.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:06 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:20:06.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:06 smithi006 ceph-mon[30108]: osdmap e457: 8 total, 8 up, 8 in 2024-03-24T12:20:06.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:06 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:20:06.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:06 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:20:06.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:06 smithi006 ceph-mon[41734]: pgmap v689: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-03-24T12:20:06.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:06 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:20:06.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:06 smithi006 ceph-mon[41734]: osdmap e457: 8 total, 8 up, 8 in 2024-03-24T12:20:06.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:06 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:20:06.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:06 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:20:07.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:07 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-24T12:20:07.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:07 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:07.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:07 smithi090 ceph-mon[33646]: osdmap e458: 8 total, 8 up, 8 in 2024-03-24T12:20:07.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:07 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:07.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:07 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-24T12:20:07.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:07 smithi090 ceph-mon[33646]: osdmap e459: 8 total, 8 up, 8 in 2024-03-24T12:20:07.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:07 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-24T12:20:07.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:07 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:07.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:07 smithi006 ceph-mon[30108]: osdmap e458: 8 total, 8 up, 8 in 2024-03-24T12:20:07.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:07 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:07.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:07 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-24T12:20:07.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:07 smithi006 ceph-mon[30108]: osdmap e459: 8 total, 8 up, 8 in 2024-03-24T12:20:07.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:07 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-24T12:20:07.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:07 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:07.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:07 smithi006 ceph-mon[41734]: osdmap e458: 8 total, 8 up, 8 in 2024-03-24T12:20:07.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:07 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:07.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:07 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-24T12:20:07.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:07 smithi006 ceph-mon[41734]: osdmap e459: 8 total, 8 up, 8 in 2024-03-24T12:20:08.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:08 smithi090 ceph-mon[33646]: pgmap v692: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:20:08.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:08 smithi006 ceph-mon[30108]: pgmap v692: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:20:08.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:08 smithi006 ceph-mon[41734]: pgmap v692: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:20:09.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:09 smithi090 ceph-mon[33646]: osdmap e460: 8 total, 8 up, 8 in 2024-03-24T12:20:09.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:09 smithi006 ceph-mon[30108]: osdmap e460: 8 total, 8 up, 8 in 2024-03-24T12:20:09.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:09 smithi006 ceph-mon[41734]: osdmap e460: 8 total, 8 up, 8 in 2024-03-24T12:20:10.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:10 smithi090 ceph-mon[33646]: pgmap v695: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:20:10.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:10 smithi090 ceph-mon[33646]: osdmap e461: 8 total, 8 up, 8 in 2024-03-24T12:20:10.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:10.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:10 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:10.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-83"}]: dispatch 2024-03-24T12:20:10.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:10 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-83"}]: dispatch 2024-03-24T12:20:10.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:10 smithi006 ceph-mon[30108]: pgmap v695: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:20:10.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:10 smithi006 ceph-mon[30108]: osdmap e461: 8 total, 8 up, 8 in 2024-03-24T12:20:10.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:10.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:10 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:10.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-83"}]: dispatch 2024-03-24T12:20:10.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:10 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-83"}]: dispatch 2024-03-24T12:20:10.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:10 smithi006 ceph-mon[41734]: pgmap v695: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:20:10.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:10 smithi006 ceph-mon[41734]: osdmap e461: 8 total, 8 up, 8 in 2024-03-24T12:20:10.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:10.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:10 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:10.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-83"}]: dispatch 2024-03-24T12:20:10.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:10 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-83"}]: dispatch 2024-03-24T12:20:11.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:11 smithi090 ceph-mon[33646]: osdmap e462: 8 total, 8 up, 8 in 2024-03-24T12:20:11.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:11 smithi006 ceph-mon[30108]: osdmap e462: 8 total, 8 up, 8 in 2024-03-24T12:20:11.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:11 smithi006 ceph-mon[41734]: osdmap e462: 8 total, 8 up, 8 in 2024-03-24T12:20:12.863 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:12 smithi006 ceph-mon[30108]: pgmap v698: 260 pgs: 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:20:12.863 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:12 smithi006 ceph-mon[30108]: osdmap e463: 8 total, 8 up, 8 in 2024-03-24T12:20:12.863 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:12 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:12.863 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:12.863 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:12 smithi006 ceph-mon[41734]: pgmap v698: 260 pgs: 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:20:12.863 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:12 smithi006 ceph-mon[41734]: osdmap e463: 8 total, 8 up, 8 in 2024-03-24T12:20:12.864 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:12 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:12.864 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:12.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:12 smithi090 ceph-mon[33646]: pgmap v698: 260 pgs: 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:20:12.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:12 smithi090 ceph-mon[33646]: osdmap e463: 8 total, 8 up, 8 in 2024-03-24T12:20:12.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:12 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:12.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:12.929 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:20:12 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:20:13.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:13 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:20:13.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:13 smithi090 ceph-mon[33646]: osdmap e464: 8 total, 8 up, 8 in 2024-03-24T12:20:13.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:13 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:20:13.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:13 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:13.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:13 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:20:13.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:13 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:13.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:13 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:20:13.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:13 smithi006 ceph-mon[30108]: osdmap e464: 8 total, 8 up, 8 in 2024-03-24T12:20:13.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:13 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:20:13.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:13 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:13.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:13 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:20:13.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:13 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:13.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:13 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:20:13.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:13 smithi006 ceph-mon[41734]: osdmap e464: 8 total, 8 up, 8 in 2024-03-24T12:20:13.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:13 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:20:13.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:13 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:13.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:13 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:20:13.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:13 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:14.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:14 smithi090 ceph-mon[33646]: pgmap v701: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-24T12:20:14.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:14 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:20:14.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:14 smithi090 ceph-mon[33646]: osdmap e465: 8 total, 8 up, 8 in 2024-03-24T12:20:14.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:14.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:14 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:14.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:14 smithi006 ceph-mon[30108]: pgmap v701: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-24T12:20:14.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:14 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:20:14.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:14 smithi006 ceph-mon[30108]: osdmap e465: 8 total, 8 up, 8 in 2024-03-24T12:20:14.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:14.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:14 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:14.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:14 smithi006 ceph-mon[41734]: pgmap v701: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-24T12:20:14.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:14 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:20:14.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:14 smithi006 ceph-mon[41734]: osdmap e465: 8 total, 8 up, 8 in 2024-03-24T12:20:14.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:14.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:14 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:15.585 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:20:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:20:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:20:15.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:15 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:20:15.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:15 smithi090 ceph-mon[33646]: osdmap e466: 8 total, 8 up, 8 in 2024-03-24T12:20:15.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:20:15.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:15 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:20:15.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:15 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:20:15.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:15 smithi006 ceph-mon[30108]: osdmap e466: 8 total, 8 up, 8 in 2024-03-24T12:20:15.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:20:15.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:15 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:20:15.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:15 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_tier","val": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:20:15.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:15 smithi006 ceph-mon[41734]: osdmap e466: 8 total, 8 up, 8 in 2024-03-24T12:20:15.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:20:15.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:15 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:20:16.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:16 smithi090 ceph-mon[33646]: pgmap v704: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-24T12:20:16.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:16 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-24T12:20:16.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:16 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:16.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:16 smithi090 ceph-mon[33646]: osdmap e467: 8 total, 8 up, 8 in 2024-03-24T12:20:16.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:16 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:16.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:16 smithi006 ceph-mon[30108]: pgmap v704: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-24T12:20:16.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:16 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-24T12:20:16.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:16 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:16.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:16 smithi006 ceph-mon[30108]: osdmap e467: 8 total, 8 up, 8 in 2024-03-24T12:20:16.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:16 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:16.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:16 smithi006 ceph-mon[41734]: pgmap v704: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-24T12:20:16.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:16 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-24T12:20:16.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:16 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:16.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:16 smithi006 ceph-mon[41734]: osdmap e467: 8 total, 8 up, 8 in 2024-03-24T12:20:16.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:16 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:17.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-24T12:20:17.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:17 smithi090 ceph-mon[33646]: osdmap e468: 8 total, 8 up, 8 in 2024-03-24T12:20:17.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-24T12:20:17.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:17 smithi006 ceph-mon[30108]: osdmap e468: 8 total, 8 up, 8 in 2024-03-24T12:20:17.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-24T12:20:17.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:17 smithi006 ceph-mon[41734]: osdmap e468: 8 total, 8 up, 8 in 2024-03-24T12:20:18.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:18 smithi090 ceph-mon[33646]: pgmap v707: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-03-24T12:20:18.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:18 smithi090 ceph-mon[33646]: osdmap e469: 8 total, 8 up, 8 in 2024-03-24T12:20:18.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:18 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:20:18.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:18 smithi006 ceph-mon[30108]: pgmap v707: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-03-24T12:20:18.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:18 smithi006 ceph-mon[30108]: osdmap e469: 8 total, 8 up, 8 in 2024-03-24T12:20:18.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:18 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:20:18.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:18 smithi006 ceph-mon[41734]: pgmap v707: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-03-24T12:20:18.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:18 smithi006 ceph-mon[41734]: osdmap e469: 8 total, 8 up, 8 in 2024-03-24T12:20:18.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:18 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:20:19.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:19 smithi090 ceph-mon[33646]: osdmap e470: 8 total, 8 up, 8 in 2024-03-24T12:20:19.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:19.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:19.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-85"}]: dispatch 2024-03-24T12:20:19.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-85"}]: dispatch 2024-03-24T12:20:19.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:19 smithi006 ceph-mon[30108]: osdmap e470: 8 total, 8 up, 8 in 2024-03-24T12:20:19.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:19.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:19.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-85"}]: dispatch 2024-03-24T12:20:19.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-85"}]: dispatch 2024-03-24T12:20:19.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:19 smithi006 ceph-mon[41734]: osdmap e470: 8 total, 8 up, 8 in 2024-03-24T12:20:19.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:19.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:19.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-85"}]: dispatch 2024-03-24T12:20:19.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-85"}]: dispatch 2024-03-24T12:20:20.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:20 smithi090 ceph-mon[33646]: pgmap v710: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-03-24T12:20:20.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:20 smithi090 ceph-mon[33646]: osdmap e471: 8 total, 8 up, 8 in 2024-03-24T12:20:20.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:20 smithi006 ceph-mon[30108]: pgmap v710: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-03-24T12:20:20.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:20 smithi006 ceph-mon[30108]: osdmap e471: 8 total, 8 up, 8 in 2024-03-24T12:20:20.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:20 smithi006 ceph-mon[41734]: pgmap v710: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-03-24T12:20:20.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:20 smithi006 ceph-mon[41734]: osdmap e471: 8 total, 8 up, 8 in 2024-03-24T12:20:21.640 INFO:tasks.workunit.client.0.smithi006.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (18080 ms) 2024-03-24T12:20:21.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-03-24T12:20:21.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22134 ms) 2024-03-24T12:20:21.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-03-24T12:20:21.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3016 ms) 2024-03-24T12:20:21.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-03-24T12:20:21.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: require_osd_release = squid 2024-03-24T12:20:21.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3039 ms) 2024-03-24T12:20:21.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-03-24T12:20:21.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3020 ms) 2024-03-24T12:20:21.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-03-24T12:20:21.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: require_osd_release = squid 2024-03-24T12:20:21.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3028 ms) 2024-03-24T12:20:21.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-03-24T12:20:21.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: require_osd_release = squid 2024-03-24T12:20:21.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4031 ms) 2024-03-24T12:20:21.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-03-24T12:20:21.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: require_osd_release = squid 2024-03-24T12:20:21.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (39400 ms) 2024-03-24T12:20:21.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-03-24T12:20:21.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: require_osd_release = squid 2024-03-24T12:20:21.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (16345 ms) 2024-03-24T12:20:21.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-03-24T12:20:21.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: require_osd_release = squid 2024-03-24T12:20:21.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4036 ms) 2024-03-24T12:20:21.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-03-24T12:20:21.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: require_osd_release = squid 2024-03-24T12:20:21.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3021 ms) 2024-03-24T12:20:21.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-03-24T12:20:21.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: require_osd_release = squid 2024-03-24T12:20:21.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (24341 ms) 2024-03-24T12:20:21.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-03-24T12:20:21.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: require_osd_release = squid 2024-03-24T12:20:21.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (14619 ms) 2024-03-24T12:20:21.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-03-24T12:20:21.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: require_osd_release = squid 2024-03-24T12:20:21.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5043 ms) 2024-03-24T12:20:21.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-03-24T12:20:21.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: require_osd_release = squid 2024-03-24T12:20:21.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4032 ms) 2024-03-24T12:20:21.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-03-24T12:20:21.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: require_osd_release = squid 2024-03-24T12:20:21.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: waiting for scrubs... 2024-03-24T12:20:21.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: done waiting 2024-03-24T12:20:21.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (25083 ms) 2024-03-24T12:20:21.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-03-24T12:20:21.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: require_osd_release = squid 2024-03-24T12:20:21.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10080 ms) 2024-03-24T12:20:21.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-03-24T12:20:21.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: require_osd_release = squid 2024-03-24T12:20:21.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9053 ms) 2024-03-24T12:20:21.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-03-24T12:20:21.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: require_osd_release = squid 2024-03-24T12:20:21.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9075 ms) 2024-03-24T12:20:21.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-03-24T12:20:21.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:21 smithi090 ceph-mon[33646]: osdmap e472: 8 total, 8 up, 8 in 2024-03-24T12:20:21.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:21.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:21 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:21.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:21 smithi006 ceph-mon[30108]: osdmap e472: 8 total, 8 up, 8 in 2024-03-24T12:20:21.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:21.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:21 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:21.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:21 smithi006 ceph-mon[41734]: osdmap e472: 8 total, 8 up, 8 in 2024-03-24T12:20:21.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:21.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:21 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:22.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:20:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:20:22.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:22 smithi090 ceph-mon[33646]: pgmap v713: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:20:22.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:22 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:20:22.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:20:22.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:22 smithi090 ceph-mon[33646]: osdmap e473: 8 total, 8 up, 8 in 2024-03-24T12:20:22.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:20:22.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:22 smithi006 ceph-mon[30108]: pgmap v713: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:20:22.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:22 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:20:22.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:20:22.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:22 smithi006 ceph-mon[30108]: osdmap e473: 8 total, 8 up, 8 in 2024-03-24T12:20:22.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:20:22.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:22 smithi006 ceph-mon[41734]: pgmap v713: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:20:22.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:22 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:20:22.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:20:22.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:22 smithi006 ceph-mon[41734]: osdmap e473: 8 total, 8 up, 8 in 2024-03-24T12:20:22.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:20:23.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:23 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:20:23.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:23 smithi090 ceph-mon[33646]: osdmap e474: 8 total, 8 up, 8 in 2024-03-24T12:20:23.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:23.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:23.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:23 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:20:23.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:23 smithi006 ceph-mon[30108]: osdmap e474: 8 total, 8 up, 8 in 2024-03-24T12:20:23.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:23.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:23.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:23 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:20:23.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:23 smithi006 ceph-mon[41734]: osdmap e474: 8 total, 8 up, 8 in 2024-03-24T12:20:23.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:23.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:24.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:24 smithi090 ceph-mon[33646]: pgmap v716: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-03-24T12:20:24.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:20:24.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:24 smithi090 ceph-mon[33646]: osdmap e475: 8 total, 8 up, 8 in 2024-03-24T12:20:24.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_tier","val": "test-rados-api-smithi006-82061-89-test-flush"}]: dispatch 2024-03-24T12:20:24.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_tier","val": "test-rados-api-smithi006-82061-89-test-flush"}]: dispatch 2024-03-24T12:20:24.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:24 smithi006 ceph-mon[30108]: pgmap v716: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-03-24T12:20:24.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:20:24.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:24 smithi006 ceph-mon[30108]: osdmap e475: 8 total, 8 up, 8 in 2024-03-24T12:20:24.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_tier","val": "test-rados-api-smithi006-82061-89-test-flush"}]: dispatch 2024-03-24T12:20:24.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_tier","val": "test-rados-api-smithi006-82061-89-test-flush"}]: dispatch 2024-03-24T12:20:24.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:24 smithi006 ceph-mon[41734]: pgmap v716: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-03-24T12:20:24.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:20:24.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:24 smithi006 ceph-mon[41734]: osdmap e475: 8 total, 8 up, 8 in 2024-03-24T12:20:24.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_tier","val": "test-rados-api-smithi006-82061-89-test-flush"}]: dispatch 2024-03-24T12:20:24.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_tier","val": "test-rados-api-smithi006-82061-89-test-flush"}]: dispatch 2024-03-24T12:20:25.658 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:20:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:20:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:20:25.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:25 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_tier","val": "test-rados-api-smithi006-82061-89-test-flush"}]': finished 2024-03-24T12:20:25.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:25 smithi090 ceph-mon[33646]: osdmap e476: 8 total, 8 up, 8 in 2024-03-24T12:20:25.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:20:25.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:25 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:20:25.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:25 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_tier","val": "test-rados-api-smithi006-82061-89-test-flush"}]': finished 2024-03-24T12:20:25.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:25 smithi006 ceph-mon[30108]: osdmap e476: 8 total, 8 up, 8 in 2024-03-24T12:20:25.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:20:25.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:25 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:20:25.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:25 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_tier","val": "test-rados-api-smithi006-82061-89-test-flush"}]': finished 2024-03-24T12:20:25.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:25 smithi006 ceph-mon[41734]: osdmap e476: 8 total, 8 up, 8 in 2024-03-24T12:20:25.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:20:25.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:25 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:20:26.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:26 smithi090 ceph-mon[33646]: pgmap v719: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-03-24T12:20:26.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-24T12:20:26.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:26.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:26 smithi090 ceph-mon[33646]: osdmap e477: 8 total, 8 up, 8 in 2024-03-24T12:20:26.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:26.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:26 smithi006 ceph-mon[30108]: pgmap v719: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-03-24T12:20:26.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-24T12:20:26.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:26.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:26 smithi006 ceph-mon[30108]: osdmap e477: 8 total, 8 up, 8 in 2024-03-24T12:20:26.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:26.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:26 smithi006 ceph-mon[41734]: pgmap v719: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-03-24T12:20:26.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-24T12:20:26.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:26.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:26 smithi006 ceph-mon[41734]: osdmap e477: 8 total, 8 up, 8 in 2024-03-24T12:20:26.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:20:27.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:27 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-24T12:20:27.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:27 smithi090 ceph-mon[33646]: osdmap e478: 8 total, 8 up, 8 in 2024-03-24T12:20:27.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:27 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-24T12:20:27.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:27 smithi006 ceph-mon[30108]: osdmap e478: 8 total, 8 up, 8 in 2024-03-24T12:20:27.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:27 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-24T12:20:27.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:27 smithi006 ceph-mon[41734]: osdmap e478: 8 total, 8 up, 8 in 2024-03-24T12:20:28.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:28 smithi006 ceph-mon[30108]: pgmap v722: 324 pgs: 324 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-03-24T12:20:28.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:28 smithi006 ceph-mon[30108]: osdmap e479: 8 total, 8 up, 8 in 2024-03-24T12:20:28.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:28.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:28.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-87"}]: dispatch 2024-03-24T12:20:28.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-87"}]: dispatch 2024-03-24T12:20:28.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:28 smithi006 ceph-mon[41734]: pgmap v722: 324 pgs: 324 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-03-24T12:20:28.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:28 smithi006 ceph-mon[41734]: osdmap e479: 8 total, 8 up, 8 in 2024-03-24T12:20:28.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:28.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:28.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-87"}]: dispatch 2024-03-24T12:20:28.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-87"}]: dispatch 2024-03-24T12:20:29.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:28 smithi090 ceph-mon[33646]: pgmap v722: 324 pgs: 324 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-03-24T12:20:29.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:28 smithi090 ceph-mon[33646]: osdmap e479: 8 total, 8 up, 8 in 2024-03-24T12:20:29.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:29.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:29.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-87"}]: dispatch 2024-03-24T12:20:29.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-87"}]: dispatch 2024-03-24T12:20:29.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:29 smithi006 ceph-mon[30108]: osdmap e480: 8 total, 8 up, 8 in 2024-03-24T12:20:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:29 smithi006 ceph-mon[30108]: pgmap v725: 260 pgs: 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:20:29.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:29 smithi006 ceph-mon[41734]: osdmap e480: 8 total, 8 up, 8 in 2024-03-24T12:20:29.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:29 smithi006 ceph-mon[41734]: pgmap v725: 260 pgs: 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:20:30.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:29 smithi090 ceph-mon[33646]: osdmap e480: 8 total, 8 up, 8 in 2024-03-24T12:20:30.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:29 smithi090 ceph-mon[33646]: pgmap v725: 260 pgs: 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:20:30.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:30 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:20:30.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:30 smithi006 ceph-mon[30108]: osdmap e481: 8 total, 8 up, 8 in 2024-03-24T12:20:30.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:30 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:30.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:30 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:30.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:30 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:20:30.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:30 smithi006 ceph-mon[41734]: osdmap e481: 8 total, 8 up, 8 in 2024-03-24T12:20:30.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:30 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:30.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:30 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:31.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:30 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:20:31.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:30 smithi090 ceph-mon[33646]: osdmap e481: 8 total, 8 up, 8 in 2024-03-24T12:20:31.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:30 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:31.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:30 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:32.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:31 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:20:32.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:31 smithi090 ceph-mon[33646]: osdmap e482: 8 total, 8 up, 8 in 2024-03-24T12:20:32.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:20:32.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:32.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:31 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:32.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:31 smithi090 ceph-mon[33646]: pgmap v728: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:20:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:31 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:20:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:31 smithi006 ceph-mon[30108]: osdmap e482: 8 total, 8 up, 8 in 2024-03-24T12:20:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:20:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:31 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:31 smithi006 ceph-mon[30108]: pgmap v728: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:20:32.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:31 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:20:32.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:31 smithi006 ceph-mon[41734]: osdmap e482: 8 total, 8 up, 8 in 2024-03-24T12:20:32.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:20:32.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:32.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:31 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:20:32.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:31 smithi006 ceph-mon[41734]: pgmap v728: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:20:32.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:32 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:20:32.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:32 smithi090 ceph-mon[33646]: osdmap e483: 8 total, 8 up, 8 in 2024-03-24T12:20:32.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:32 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:20:32.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:20:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:20:33.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:32 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:20:33.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:32 smithi006 ceph-mon[30108]: osdmap e483: 8 total, 8 up, 8 in 2024-03-24T12:20:33.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:32 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:20:33.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:32 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:20:33.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:32 smithi006 ceph-mon[41734]: osdmap e483: 8 total, 8 up, 8 in 2024-03-24T12:20:33.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:32 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:20:34.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:33 smithi090 ceph-mon[33646]: osdmap e484: 8 total, 8 up, 8 in 2024-03-24T12:20:34.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:33 smithi090 ceph-mon[33646]: pgmap v731: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-03-24T12:20:34.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:33 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:20:34.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:33 smithi006 ceph-mon[30108]: osdmap e484: 8 total, 8 up, 8 in 2024-03-24T12:20:34.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:33 smithi006 ceph-mon[30108]: pgmap v731: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-03-24T12:20:34.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:33 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:20:34.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:33 smithi006 ceph-mon[41734]: osdmap e484: 8 total, 8 up, 8 in 2024-03-24T12:20:34.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:33 smithi006 ceph-mon[41734]: pgmap v731: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-03-24T12:20:34.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:33 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:20:35.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:34 smithi090 ceph-mon[33646]: osdmap e485: 8 total, 8 up, 8 in 2024-03-24T12:20:35.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:35.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:34 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:35.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-90"}]: dispatch 2024-03-24T12:20:35.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:34 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-90"}]: dispatch 2024-03-24T12:20:35.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:34 smithi006 ceph-mon[30108]: osdmap e485: 8 total, 8 up, 8 in 2024-03-24T12:20:35.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:35.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:34 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:35.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-90"}]: dispatch 2024-03-24T12:20:35.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:34 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-90"}]: dispatch 2024-03-24T12:20:35.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:34 smithi006 ceph-mon[41734]: osdmap e485: 8 total, 8 up, 8 in 2024-03-24T12:20:35.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:35.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:34 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:35.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-90"}]: dispatch 2024-03-24T12:20:35.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:34 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-90"}]: dispatch 2024-03-24T12:20:35.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:20:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:20:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:20:36.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:35 smithi090 ceph-mon[33646]: osdmap e486: 8 total, 8 up, 8 in 2024-03-24T12:20:36.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:35 smithi090 ceph-mon[33646]: pgmap v734: 260 pgs: 260 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:20:36.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:35 smithi006 ceph-mon[30108]: osdmap e486: 8 total, 8 up, 8 in 2024-03-24T12:20:36.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:35 smithi006 ceph-mon[30108]: pgmap v734: 260 pgs: 260 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:20:36.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:35 smithi006 ceph-mon[41734]: osdmap e486: 8 total, 8 up, 8 in 2024-03-24T12:20:36.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:35 smithi006 ceph-mon[41734]: pgmap v734: 260 pgs: 260 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:20:37.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:36 smithi090 ceph-mon[33646]: osdmap e487: 8 total, 8 up, 8 in 2024-03-24T12:20:37.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:36 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:37.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:36 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:37.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:36 smithi006 ceph-mon[30108]: osdmap e487: 8 total, 8 up, 8 in 2024-03-24T12:20:37.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:36 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:37.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:36 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:37.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:36 smithi006 ceph-mon[41734]: osdmap e487: 8 total, 8 up, 8 in 2024-03-24T12:20:37.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:36 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:37.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:36 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:37 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:20:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:37 smithi090 ceph-mon[33646]: osdmap e488: 8 total, 8 up, 8 in 2024-03-24T12:20:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:20:38.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:37 smithi090 ceph-mon[33646]: pgmap v737: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:20:38.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:37 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:20:38.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:37 smithi006 ceph-mon[30108]: osdmap e488: 8 total, 8 up, 8 in 2024-03-24T12:20:38.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:20:38.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:37 smithi006 ceph-mon[30108]: pgmap v737: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:20:38.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:37 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:20:38.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:37 smithi006 ceph-mon[41734]: osdmap e488: 8 total, 8 up, 8 in 2024-03-24T12:20:38.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:20:38.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:37 smithi006 ceph-mon[41734]: pgmap v737: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:20:39.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:38 smithi090 ceph-mon[33646]: osdmap e489: 8 total, 8 up, 8 in 2024-03-24T12:20:39.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:38 smithi006 ceph-mon[30108]: osdmap e489: 8 total, 8 up, 8 in 2024-03-24T12:20:39.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:38 smithi006 ceph-mon[41734]: osdmap e489: 8 total, 8 up, 8 in 2024-03-24T12:20:40.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:39 smithi090 ceph-mon[33646]: osdmap e490: 8 total, 8 up, 8 in 2024-03-24T12:20:40.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:39 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:40.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:40.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:39 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-92"}]: dispatch 2024-03-24T12:20:40.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-92"}]: dispatch 2024-03-24T12:20:40.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:39 smithi090 ceph-mon[33646]: pgmap v740: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-03-24T12:20:40.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:39 smithi006 ceph-mon[30108]: osdmap e490: 8 total, 8 up, 8 in 2024-03-24T12:20:40.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:40.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:40.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-92"}]: dispatch 2024-03-24T12:20:40.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-92"}]: dispatch 2024-03-24T12:20:40.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:39 smithi006 ceph-mon[30108]: pgmap v740: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-03-24T12:20:40.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:39 smithi006 ceph-mon[41734]: osdmap e490: 8 total, 8 up, 8 in 2024-03-24T12:20:40.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:39 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:40.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:20:40.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:39 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-92"}]: dispatch 2024-03-24T12:20:40.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-92"}]: dispatch 2024-03-24T12:20:40.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:39 smithi006 ceph-mon[41734]: pgmap v740: 292 pgs: 292 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-03-24T12:20:41.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:40 smithi090 ceph-mon[33646]: osdmap e491: 8 total, 8 up, 8 in 2024-03-24T12:20:41.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:40 smithi006 ceph-mon[30108]: osdmap e491: 8 total, 8 up, 8 in 2024-03-24T12:20:41.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:40 smithi006 ceph-mon[41734]: osdmap e491: 8 total, 8 up, 8 in 2024-03-24T12:20:42.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:41 smithi006 ceph-mon[30108]: osdmap e492: 8 total, 8 up, 8 in 2024-03-24T12:20:42.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:42.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:41 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:42.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:41 smithi006 ceph-mon[30108]: pgmap v743: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:20:42.113 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:41 smithi006 ceph-mon[41734]: osdmap e492: 8 total, 8 up, 8 in 2024-03-24T12:20:42.113 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:42.113 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:41 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:42.113 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:41 smithi006 ceph-mon[41734]: pgmap v743: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:20:42.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:41 smithi090 ceph-mon[33646]: osdmap e492: 8 total, 8 up, 8 in 2024-03-24T12:20:42.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:42.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:41 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:20:42.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:41 smithi090 ceph-mon[33646]: pgmap v743: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:20:42.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:42 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:20:42.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:42 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:20:42.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:42 smithi090 ceph-mon[33646]: osdmap e493: 8 total, 8 up, 8 in 2024-03-24T12:20:42.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:42 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:20:42.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:42 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:20:42.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:20:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:20:43.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:42 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:20:43.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:42 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:20:43.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:42 smithi006 ceph-mon[30108]: osdmap e493: 8 total, 8 up, 8 in 2024-03-24T12:20:43.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:42 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:20:43.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:42 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:20:43.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:42 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:20:43.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:42 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:20:43.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:42 smithi006 ceph-mon[41734]: osdmap e493: 8 total, 8 up, 8 in 2024-03-24T12:20:43.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:42 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:20:43.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:42 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:20:44.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:43 smithi090 ceph-mon[33646]: osdmap e494: 8 total, 8 up, 8 in 2024-03-24T12:20:44.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:43 smithi090 ceph-mon[33646]: pgmap v746: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-24T12:20:44.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:43 smithi006 ceph-mon[30108]: osdmap e494: 8 total, 8 up, 8 in 2024-03-24T12:20:44.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:43 smithi006 ceph-mon[30108]: pgmap v746: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-24T12:20:44.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:43 smithi006 ceph-mon[41734]: osdmap e494: 8 total, 8 up, 8 in 2024-03-24T12:20:44.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:43 smithi006 ceph-mon[41734]: pgmap v746: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-24T12:20:45.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:44 smithi090 ceph-mon[33646]: osdmap e495: 8 total, 8 up, 8 in 2024-03-24T12:20:45.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:44 smithi006 ceph-mon[30108]: osdmap e495: 8 total, 8 up, 8 in 2024-03-24T12:20:45.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:44 smithi006 ceph-mon[41734]: osdmap e495: 8 total, 8 up, 8 in 2024-03-24T12:20:45.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:20:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:20:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:20:46.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:45 smithi090 ceph-mon[33646]: pgmap v748: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-03-24T12:20:46.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:45 smithi006 ceph-mon[30108]: pgmap v748: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-03-24T12:20:46.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:45 smithi006 ceph-mon[41734]: pgmap v748: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-03-24T12:20:48.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:48 smithi090 ceph-mon[33646]: pgmap v749: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-24T12:20:48.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:48 smithi006 ceph-mon[30108]: pgmap v749: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-24T12:20:48.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:48 smithi006 ceph-mon[41734]: pgmap v749: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-24T12:20:49.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:49 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:20:49.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:49 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:20:49.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:49 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:20:50.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:50 smithi090 ceph-mon[33646]: pgmap v750: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-03-24T12:20:50.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:50 smithi006 ceph-mon[30108]: pgmap v750: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-03-24T12:20:50.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:50 smithi006 ceph-mon[41734]: pgmap v750: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-03-24T12:20:51.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:51 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:20:51.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:51 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:20:51.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:51 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:20:52.606 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:52 smithi090 ceph-mon[33646]: pgmap v751: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 243 B/s wr, 2 op/s 2024-03-24T12:20:52.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:52 smithi006 ceph-mon[30108]: pgmap v751: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 243 B/s wr, 2 op/s 2024-03-24T12:20:52.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:52 smithi006 ceph-mon[41734]: pgmap v751: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 243 B/s wr, 2 op/s 2024-03-24T12:20:52.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:20:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:20:53.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:53 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:20:53.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:53 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:20:53.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:53 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:20:54.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:54 smithi090 ceph-mon[33646]: pgmap v752: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2024-03-24T12:20:54.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:54 smithi006 ceph-mon[30108]: pgmap v752: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2024-03-24T12:20:54.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:54 smithi006 ceph-mon[41734]: pgmap v752: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2024-03-24T12:20:55.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:55 smithi090 ceph-mon[33646]: osdmap e496: 8 total, 8 up, 8 in 2024-03-24T12:20:55.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:55 smithi006 ceph-mon[30108]: osdmap e496: 8 total, 8 up, 8 in 2024-03-24T12:20:55.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:20:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:20:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:20:55.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:55 smithi006 ceph-mon[41734]: osdmap e496: 8 total, 8 up, 8 in 2024-03-24T12:20:56.579 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:56 smithi090 ceph-mon[33646]: pgmap v754: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2024-03-24T12:20:56.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:56 smithi006 ceph-mon[30108]: pgmap v754: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2024-03-24T12:20:56.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:56 smithi006 ceph-mon[41734]: pgmap v754: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 204 B/s wr, 2 op/s 2024-03-24T12:20:58.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:20:58 smithi090 ceph-mon[33646]: pgmap v755: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:20:58.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:20:58 smithi006 ceph-mon[30108]: pgmap v755: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:20:58.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:20:58 smithi006 ceph-mon[41734]: pgmap v755: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:21:00.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:00 smithi090 ceph-mon[33646]: pgmap v756: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:21:00.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:00 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:21:00.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:00 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:00 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:00 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:00 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:00 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:00 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:00 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:00 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:00 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:00 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:00 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:00 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:00 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:00 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:00 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:00 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:00 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:21:00.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:00 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:21:00.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:00 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:21:00.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:00 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:21:00.680 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:00 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:21:00.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[30108]: pgmap v756: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:21:00.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:21:00.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:21:00.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:21:00.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:21:00.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:21:00.712 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:21:00.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[41734]: pgmap v756: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:21:00.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:21:00.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-24T12:21:00.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:21:00.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:21:00.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:21:00.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:21:00.714 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:00 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:21:01.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:01 smithi090 ceph-mon[33646]: Adjusting osd_memory_target on smithi090 to 3248M 2024-03-24T12:21:01.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:01 smithi090 ceph-mon[33646]: Adjusting osd_memory_target on smithi006 to 3249M 2024-03-24T12:21:01.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:01 smithi090 ceph-mon[33646]: osdmap e497: 8 total, 8 up, 8 in 2024-03-24T12:21:01.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:01 smithi006 ceph-mon[30108]: Adjusting osd_memory_target on smithi090 to 3248M 2024-03-24T12:21:01.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:01 smithi006 ceph-mon[30108]: Adjusting osd_memory_target on smithi006 to 3249M 2024-03-24T12:21:01.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:01 smithi006 ceph-mon[30108]: osdmap e497: 8 total, 8 up, 8 in 2024-03-24T12:21:01.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:01 smithi006 ceph-mon[41734]: Adjusting osd_memory_target on smithi090 to 3248M 2024-03-24T12:21:01.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:01 smithi006 ceph-mon[41734]: Adjusting osd_memory_target on smithi006 to 3249M 2024-03-24T12:21:01.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:01 smithi006 ceph-mon[41734]: osdmap e497: 8 total, 8 up, 8 in 2024-03-24T12:21:02.617 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:02 smithi090 ceph-mon[33646]: pgmap v758: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:21:02.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:02 smithi006 ceph-mon[30108]: pgmap v758: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:21:02.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:02 smithi006 ceph-mon[41734]: pgmap v758: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:21:02.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:21:02 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:21:03.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:03 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:21:03.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:03 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:21:03.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:03 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:21:04.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:04 smithi090 ceph-mon[33646]: pgmap v759: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:21:04.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:21:04.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:04 smithi006 ceph-mon[30108]: pgmap v759: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:21:04.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:21:04.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:04 smithi006 ceph-mon[41734]: pgmap v759: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:21:04.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:21:05.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:05 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:05.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:05 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:05.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:05 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-94"}]: dispatch 2024-03-24T12:21:05.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:05 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-94"}]: dispatch 2024-03-24T12:21:05.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:21:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:21:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:21:05.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:05 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:05.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:05 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:05.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:05 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-94"}]: dispatch 2024-03-24T12:21:05.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:05 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-94"}]: dispatch 2024-03-24T12:21:05.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:05 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:05.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:05 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:05.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:05 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-94"}]: dispatch 2024-03-24T12:21:05.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:05 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-94"}]: dispatch 2024-03-24T12:21:06.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:06 smithi090 ceph-mon[33646]: pgmap v760: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:21:06.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:06 smithi090 ceph-mon[33646]: osdmap e498: 8 total, 8 up, 8 in 2024-03-24T12:21:06.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:06 smithi006 ceph-mon[30108]: pgmap v760: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:21:06.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:06 smithi006 ceph-mon[30108]: osdmap e498: 8 total, 8 up, 8 in 2024-03-24T12:21:06.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:06 smithi006 ceph-mon[41734]: pgmap v760: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:21:06.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:06 smithi006 ceph-mon[41734]: osdmap e498: 8 total, 8 up, 8 in 2024-03-24T12:21:07.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:07 smithi090 ceph-mon[33646]: osdmap e499: 8 total, 8 up, 8 in 2024-03-24T12:21:07.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:07 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:07.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:07 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:07.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:07 smithi006 ceph-mon[30108]: osdmap e499: 8 total, 8 up, 8 in 2024-03-24T12:21:07.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:07 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:07.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:07 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:07.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:07 smithi006 ceph-mon[41734]: osdmap e499: 8 total, 8 up, 8 in 2024-03-24T12:21:07.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:07 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:07.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:07 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:08.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:08 smithi090 ceph-mon[33646]: pgmap v763: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:21:08.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:08 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:21:08.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:08 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:21:08.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:08 smithi090 ceph-mon[33646]: osdmap e500: 8 total, 8 up, 8 in 2024-03-24T12:21:08.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:21:08.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:08 smithi006 ceph-mon[30108]: pgmap v763: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:21:08.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:08 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:21:08.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:08 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:21:08.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:08 smithi006 ceph-mon[30108]: osdmap e500: 8 total, 8 up, 8 in 2024-03-24T12:21:08.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:21:08.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:08 smithi006 ceph-mon[41734]: pgmap v763: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:21:08.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:08 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:21:08.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:08 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:21:08.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:08 smithi006 ceph-mon[41734]: osdmap e500: 8 total, 8 up, 8 in 2024-03-24T12:21:08.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:21:09.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:09 smithi090 ceph-mon[33646]: osdmap e501: 8 total, 8 up, 8 in 2024-03-24T12:21:09.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:09 smithi006 ceph-mon[30108]: osdmap e501: 8 total, 8 up, 8 in 2024-03-24T12:21:09.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:09 smithi006 ceph-mon[41734]: osdmap e501: 8 total, 8 up, 8 in 2024-03-24T12:21:10.178 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:21:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=cleanup t=2024-03-24T12:21:09.789245258Z level=info msg="Completed cleanup jobs" duration=2.282246ms 2024-03-24T12:21:10.178 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:21:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=sqlstore.transactions t=2024-03-24T12:21:09.797537551Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-03-24T12:21:10.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:10 smithi090 ceph-mon[33646]: pgmap v766: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:21:10.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:10 smithi006 ceph-mon[30108]: pgmap v766: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:21:10.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:10 smithi006 ceph-mon[41734]: pgmap v766: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:21:12.621 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:12 smithi090 ceph-mon[33646]: pgmap v767: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 869 B/s rd, 173 B/s wr, 1 op/s 2024-03-24T12:21:12.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:12 smithi006 ceph-mon[30108]: pgmap v767: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 869 B/s rd, 173 B/s wr, 1 op/s 2024-03-24T12:21:12.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:12 smithi006 ceph-mon[41734]: pgmap v767: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 869 B/s rd, 173 B/s wr, 1 op/s 2024-03-24T12:21:12.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:21:12 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:21:13.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:13 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:21:13.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:13 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:21:13.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:13 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:21:14.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:14 smithi090 ceph-mon[33646]: pgmap v768: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 893 B/s wr, 3 op/s 2024-03-24T12:21:14.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:14 smithi006 ceph-mon[30108]: pgmap v768: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 893 B/s wr, 3 op/s 2024-03-24T12:21:14.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:14 smithi006 ceph-mon[41734]: pgmap v768: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 893 B/s wr, 3 op/s 2024-03-24T12:21:15.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:21:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:21:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:21:16.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:16 smithi090 ceph-mon[33646]: pgmap v769: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-03-24T12:21:16.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:16 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:21:16.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:16 smithi006 ceph-mon[30108]: pgmap v769: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-03-24T12:21:16.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:16 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:21:16.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:16 smithi006 ceph-mon[41734]: pgmap v769: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-03-24T12:21:16.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:16 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:21:18.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:18 smithi090 ceph-mon[33646]: pgmap v770: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 518 B/s wr, 2 op/s 2024-03-24T12:21:18.678 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:21:18 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:21:18.388+0000 7f1d32d4b700 -1 snap_mapper.add_oid found existing snaps mapped on 113:1534c4e0:test-rados-api-smithi006-82061-97::foo:21, removing 2024-03-24T12:21:18.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:18 smithi006 ceph-mon[30108]: pgmap v770: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 518 B/s wr, 2 op/s 2024-03-24T12:21:18.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:18 smithi006 ceph-mon[41734]: pgmap v770: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 518 B/s wr, 2 op/s 2024-03-24T12:21:18.711 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:21:18 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-0[46125]: 2024-03-24T12:21:18.388+0000 7ff5d152b700 -1 snap_mapper.add_oid found existing snaps mapped on 113:1534c4e0:test-rados-api-smithi006-82061-97::foo:21, removing 2024-03-24T12:21:18.711 INFO:journalctl@ceph.osd.3.smithi006.stdout:Mar 24 12:21:18 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-3[64519]: 2024-03-24T12:21:18.388+0000 7f3097880700 -1 snap_mapper.add_oid found existing snaps mapped on 113:1534c4e0:test-rados-api-smithi006-82061-97::foo:21, removing 2024-03-24T12:21:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-96"}]: dispatch 2024-03-24T12:21:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-96"}]: dispatch 2024-03-24T12:21:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:21:19.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:19.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-96"}]: dispatch 2024-03-24T12:21:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-96"}]: dispatch 2024-03-24T12:21:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:21:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-96"}]: dispatch 2024-03-24T12:21:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-96"}]: dispatch 2024-03-24T12:21:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:21:20.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:20 smithi090 ceph-mon[33646]: pgmap v771: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 942 B/s rd, 471 B/s wr, 1 op/s 2024-03-24T12:21:20.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:20 smithi090 ceph-mon[33646]: osdmap e502: 8 total, 8 up, 8 in 2024-03-24T12:21:20.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:20 smithi006 ceph-mon[30108]: pgmap v771: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 942 B/s rd, 471 B/s wr, 1 op/s 2024-03-24T12:21:20.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:20 smithi006 ceph-mon[30108]: osdmap e502: 8 total, 8 up, 8 in 2024-03-24T12:21:20.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:20 smithi006 ceph-mon[41734]: pgmap v771: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 942 B/s rd, 471 B/s wr, 1 op/s 2024-03-24T12:21:20.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:20 smithi006 ceph-mon[41734]: osdmap e502: 8 total, 8 up, 8 in 2024-03-24T12:21:21.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:21 smithi090 ceph-mon[33646]: osdmap e503: 8 total, 8 up, 8 in 2024-03-24T12:21:21.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:21.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:21 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:21.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:21 smithi006 ceph-mon[30108]: osdmap e503: 8 total, 8 up, 8 in 2024-03-24T12:21:21.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:21.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:21 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:21.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:21 smithi006 ceph-mon[41734]: osdmap e503: 8 total, 8 up, 8 in 2024-03-24T12:21:21.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:21.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:21 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:22.557 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc7750 2024-03-24T12:21:22.557 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc7880 2024-03-24T12:21:22.557 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc7bd0 2024-03-24T12:21:22.557 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc8060 2024-03-24T12:21:22.557 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc81e0 2024-03-24T12:21:22.557 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc7990 2024-03-24T12:21:22.557 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc8330 2024-03-24T12:21:22.557 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc87e0 2024-03-24T12:21:22.558 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc88f0 2024-03-24T12:21:22.558 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc6a50 2024-03-24T12:21:22.558 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc6ba0 2024-03-24T12:21:22.558 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc7ce0 2024-03-24T12:21:22.558 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc7e10 2024-03-24T12:21:22.558 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc7f20 2024-03-24T12:21:22.558 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc8a40 2024-03-24T12:21:22.558 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc8ed0 2024-03-24T12:21:22.558 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc9020 2024-03-24T12:21:22.558 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc8b50 2024-03-24T12:21:22.558 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc8ca0 2024-03-24T12:21:22.558 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc9610 2024-03-24T12:21:22.558 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc9170 2024-03-24T12:21:22.559 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc92a0 2024-03-24T12:21:22.559 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc93f0 2024-03-24T12:21:22.559 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc9740 2024-03-24T12:21:22.559 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc9bd0 2024-03-24T12:21:22.559 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc9d00 2024-03-24T12:21:22.559 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc9850 2024-03-24T12:21:22.559 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc99a0 2024-03-24T12:21:22.559 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adca2d0 2024-03-24T12:21:22.559 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc9e30 2024-03-24T12:21:22.559 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc9f80 2024-03-24T12:21:22.559 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adca0b0 2024-03-24T12:21:22.559 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adca420 2024-03-24T12:21:22.559 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adca8b0 2024-03-24T12:21:22.559 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adcad40 2024-03-24T12:21:22.560 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adcae70 2024-03-24T12:21:22.560 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adca9c0 2024-03-24T12:21:22.560 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adcab10 2024-03-24T12:21:22.560 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc8440 2024-03-24T12:21:22.560 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc8570 2024-03-24T12:21:22.560 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adcac20 2024-03-24T12:21:22.560 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adc86a0 2024-03-24T12:21:22.560 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adcb440 2024-03-24T12:21:22.560 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adcafa0 2024-03-24T12:21:22.560 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adcb0f0 2024-03-24T12:21:22.560 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adcb200 2024-03-24T12:21:22.560 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adcb330 2024-03-24T12:21:22.560 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adcba30 2024-03-24T12:21:22.560 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adcb590 2024-03-24T12:21:22.560 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adcb6c0 2024-03-24T12:21:22.560 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 0x55ac4adcb7d0 2024-03-24T12:21:22.561 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.561 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.561 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.561 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.561 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.561 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.561 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.561 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.561 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.561 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.561 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.561 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.561 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.561 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.562 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.562 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.562 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.562 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.562 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.562 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.562 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.562 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.562 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.562 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.562 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.562 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.562 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.562 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.562 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.563 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.563 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.563 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.563 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.563 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.563 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.563 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.563 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.563 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:21:22.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:21:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:21:22.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:22 smithi090 ceph-mon[33646]: pgmap v774: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T12:21:22.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:22 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:21:22.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:21:22.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:22 smithi090 ceph-mon[33646]: osdmap e504: 8 total, 8 up, 8 in 2024-03-24T12:21:22.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:21:22.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:21:22.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:22 smithi006 ceph-mon[30108]: pgmap v774: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T12:21:22.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:22 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:21:22.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:21:22.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:22 smithi006 ceph-mon[30108]: osdmap e504: 8 total, 8 up, 8 in 2024-03-24T12:21:22.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:21:22.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:21:22.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:22 smithi006 ceph-mon[41734]: pgmap v774: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T12:21:22.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:22 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:21:22.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:21:22.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:22 smithi006 ceph-mon[41734]: osdmap e504: 8 total, 8 up, 8 in 2024-03-24T12:21:22.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:21:22.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:21:23.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-98", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:21:23.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:23 smithi090 ceph-mon[33646]: osdmap e505: 8 total, 8 up, 8 in 2024-03-24T12:21:23.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-98", "mode": "writeback"}]: dispatch 2024-03-24T12:21:23.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-98", "mode": "writeback"}]: dispatch 2024-03-24T12:21:23.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:23 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:21:23.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-98", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:21:23.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:23 smithi006 ceph-mon[30108]: osdmap e505: 8 total, 8 up, 8 in 2024-03-24T12:21:23.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-98", "mode": "writeback"}]: dispatch 2024-03-24T12:21:23.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-98", "mode": "writeback"}]: dispatch 2024-03-24T12:21:23.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:23 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:21:23.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-98", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:21:23.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:23 smithi006 ceph-mon[41734]: osdmap e505: 8 total, 8 up, 8 in 2024-03-24T12:21:23.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-98", "mode": "writeback"}]: dispatch 2024-03-24T12:21:23.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-98", "mode": "writeback"}]: dispatch 2024-03-24T12:21:23.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:23 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:21:24.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:24 smithi090 ceph-mon[33646]: pgmap v777: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-03-24T12:21:24.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:24 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:21:24.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-98", "mode": "writeback"}]': finished 2024-03-24T12:21:24.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-98"}]: dispatch 2024-03-24T12:21:24.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:24 smithi090 ceph-mon[33646]: osdmap e506: 8 total, 8 up, 8 in 2024-03-24T12:21:24.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-98"}]: dispatch 2024-03-24T12:21:24.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:24 smithi006 ceph-mon[30108]: pgmap v777: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-03-24T12:21:24.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:24 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:21:24.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-98", "mode": "writeback"}]': finished 2024-03-24T12:21:24.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-98"}]: dispatch 2024-03-24T12:21:24.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:24 smithi006 ceph-mon[30108]: osdmap e506: 8 total, 8 up, 8 in 2024-03-24T12:21:24.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-98"}]: dispatch 2024-03-24T12:21:24.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:24 smithi006 ceph-mon[41734]: pgmap v777: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-03-24T12:21:24.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:24 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:21:24.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-98", "mode": "writeback"}]': finished 2024-03-24T12:21:24.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-98"}]: dispatch 2024-03-24T12:21:24.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:24 smithi006 ceph-mon[41734]: osdmap e506: 8 total, 8 up, 8 in 2024-03-24T12:21:24.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-98"}]: dispatch 2024-03-24T12:21:25.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:21:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:21:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:21:25.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:25 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-98"}]': finished 2024-03-24T12:21:25.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:21:25.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:25 smithi006 ceph-mon[30108]: osdmap e507: 8 total, 8 up, 8 in 2024-03-24T12:21:25.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:25 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:21:25.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:25 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-98"}]': finished 2024-03-24T12:21:25.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:21:25.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:25 smithi006 ceph-mon[41734]: osdmap e507: 8 total, 8 up, 8 in 2024-03-24T12:21:25.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:25 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:21:25.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:25 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-98"}]': finished 2024-03-24T12:21:25.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:21:25.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:25 smithi090 ceph-mon[33646]: osdmap e507: 8 total, 8 up, 8 in 2024-03-24T12:21:25.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:25 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:21:26.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:26 smithi006 ceph-mon[30108]: pgmap v780: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:21:26.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:26 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:21:26.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_type","val": "bloom"}]': finished 2024-03-24T12:21:26.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:26 smithi006 ceph-mon[30108]: osdmap e508: 8 total, 8 up, 8 in 2024-03-24T12:21:26.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-24T12:21:26.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:26 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-24T12:21:26.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:26 smithi006 ceph-mon[41734]: pgmap v780: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:21:26.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:26 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:21:26.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_type","val": "bloom"}]': finished 2024-03-24T12:21:26.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:26 smithi006 ceph-mon[41734]: osdmap e508: 8 total, 8 up, 8 in 2024-03-24T12:21:26.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-24T12:21:26.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:26 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-24T12:21:26.829 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:26 smithi090 ceph-mon[33646]: pgmap v780: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-24T12:21:26.829 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:26 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:21:26.829 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_type","val": "bloom"}]': finished 2024-03-24T12:21:26.829 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:26 smithi090 ceph-mon[33646]: osdmap e508: 8 total, 8 up, 8 in 2024-03-24T12:21:26.829 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-24T12:21:26.829 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:26 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-24T12:21:27.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:27 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_count","val": "1"}]': finished 2024-03-24T12:21:27.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:27 smithi006 ceph-mon[30108]: osdmap e509: 8 total, 8 up, 8 in 2024-03-24T12:21:27.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:27 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:21:27.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:27 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:21:27.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:27 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_count","val": "1"}]': finished 2024-03-24T12:21:27.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:27 smithi006 ceph-mon[41734]: osdmap e509: 8 total, 8 up, 8 in 2024-03-24T12:21:27.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:27 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:21:27.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:27 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:21:27.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:27 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_count","val": "1"}]': finished 2024-03-24T12:21:27.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:27 smithi090 ceph-mon[33646]: osdmap e509: 8 total, 8 up, 8 in 2024-03-24T12:21:27.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:27 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:21:27.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:27 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:21:28.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:28 smithi006 ceph-mon[30108]: pgmap v783: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:21:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:28 smithi006 ceph-mon[30108]: osdmap e510: 8 total, 8 up, 8 in 2024-03-24T12:21:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-24T12:21:28.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:28 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-24T12:21:28.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:28 smithi006 ceph-mon[41734]: pgmap v783: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:28.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:21:28.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:28 smithi006 ceph-mon[41734]: osdmap e510: 8 total, 8 up, 8 in 2024-03-24T12:21:28.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-24T12:21:28.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:28 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-24T12:21:28.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:28 smithi090 ceph-mon[33646]: pgmap v783: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:28.931 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:21:28.931 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:28 smithi090 ceph-mon[33646]: osdmap e510: 8 total, 8 up, 8 in 2024-03-24T12:21:28.931 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-24T12:21:28.931 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:28 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-24T12:21:29.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "target_max_objects","val": "250"}]': finished 2024-03-24T12:21:29.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:29 smithi090 ceph-mon[33646]: osdmap e511: 8 total, 8 up, 8 in 2024-03-24T12:21:29.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:29 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:29.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:29 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:29.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "target_max_objects","val": "250"}]': finished 2024-03-24T12:21:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:29 smithi006 ceph-mon[30108]: osdmap e511: 8 total, 8 up, 8 in 2024-03-24T12:21:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:29 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:29 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:29.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-98","var": "target_max_objects","val": "250"}]': finished 2024-03-24T12:21:29.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:29 smithi006 ceph-mon[41734]: osdmap e511: 8 total, 8 up, 8 in 2024-03-24T12:21:29.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:29 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:29.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:29 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:30 smithi090 ceph-mon[33646]: pgmap v786: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:30 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:21:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:30 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-98"}]: dispatch 2024-03-24T12:21:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:30 smithi090 ceph-mon[33646]: osdmap e512: 8 total, 8 up, 8 in 2024-03-24T12:21:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:30 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-98"}]: dispatch 2024-03-24T12:21:30.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:30 smithi006 ceph-mon[41734]: pgmap v786: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:30.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:30 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:21:30.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:30 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-98"}]: dispatch 2024-03-24T12:21:30.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:30 smithi006 ceph-mon[41734]: osdmap e512: 8 total, 8 up, 8 in 2024-03-24T12:21:30.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:30 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-98"}]: dispatch 2024-03-24T12:21:30.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:30 smithi006 ceph-mon[30108]: pgmap v786: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:30.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:30 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:21:30.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:30 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-98"}]: dispatch 2024-03-24T12:21:30.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:30 smithi006 ceph-mon[30108]: osdmap e512: 8 total, 8 up, 8 in 2024-03-24T12:21:30.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:30 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-98"}]: dispatch 2024-03-24T12:21:31.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:31 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-98"}]': finished 2024-03-24T12:21:31.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:31 smithi090 ceph-mon[33646]: osdmap e513: 8 total, 8 up, 8 in 2024-03-24T12:21:31.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:31 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:21:31.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:31 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-98"}]': finished 2024-03-24T12:21:31.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:31 smithi006 ceph-mon[30108]: osdmap e513: 8 total, 8 up, 8 in 2024-03-24T12:21:31.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:31 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:21:31.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:31 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-98"}]': finished 2024-03-24T12:21:31.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:31 smithi006 ceph-mon[41734]: osdmap e513: 8 total, 8 up, 8 in 2024-03-24T12:21:31.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:31 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:21:32.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:32 smithi090 ceph-mon[33646]: pgmap v789: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:21:32.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:32 smithi090 ceph-mon[33646]: osdmap e514: 8 total, 8 up, 8 in 2024-03-24T12:21:32.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:21:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:21:32.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:32 smithi006 ceph-mon[30108]: pgmap v789: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:21:32.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:32 smithi006 ceph-mon[30108]: osdmap e514: 8 total, 8 up, 8 in 2024-03-24T12:21:32.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:32 smithi006 ceph-mon[41734]: pgmap v789: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:21:32.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:32 smithi006 ceph-mon[41734]: osdmap e514: 8 total, 8 up, 8 in 2024-03-24T12:21:33.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:33 smithi090 ceph-mon[33646]: osdmap e515: 8 total, 8 up, 8 in 2024-03-24T12:21:33.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:33.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:33 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:33.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:33 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:21:33.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:33 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:21:33.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:33 smithi006 ceph-mon[30108]: osdmap e515: 8 total, 8 up, 8 in 2024-03-24T12:21:33.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:33.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:33 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:33.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:33 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:21:33.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:33 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:21:33.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:33 smithi006 ceph-mon[41734]: osdmap e515: 8 total, 8 up, 8 in 2024-03-24T12:21:33.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:33.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:33 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:33.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:33 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:21:33.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:33 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:21:34.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:34 smithi090 ceph-mon[33646]: pgmap v792: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:34.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:34 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:21:34.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:34 smithi090 ceph-mon[33646]: osdmap e516: 8 total, 8 up, 8 in 2024-03-24T12:21:34.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:21:34.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:34 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:21:34.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:34 smithi006 ceph-mon[30108]: pgmap v792: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:34.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:34 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:21:34.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:34 smithi006 ceph-mon[30108]: osdmap e516: 8 total, 8 up, 8 in 2024-03-24T12:21:34.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:21:34.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:34 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:21:34.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:34 smithi006 ceph-mon[41734]: pgmap v792: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:34.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:34 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:21:34.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:34 smithi006 ceph-mon[41734]: osdmap e516: 8 total, 8 up, 8 in 2024-03-24T12:21:34.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:21:34.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:34 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:21:35.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:21:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:21:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:21:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-100", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:21:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:35 smithi006 ceph-mon[30108]: osdmap e517: 8 total, 8 up, 8 in 2024-03-24T12:21:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:35 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-100"}]: dispatch 2024-03-24T12:21:35.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:35 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-100"}]: dispatch 2024-03-24T12:21:35.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-100", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:21:35.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:35 smithi006 ceph-mon[41734]: osdmap e517: 8 total, 8 up, 8 in 2024-03-24T12:21:35.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:35 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-100"}]: dispatch 2024-03-24T12:21:35.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:35 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-100"}]: dispatch 2024-03-24T12:21:35.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-100", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:21:35.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:35 smithi090 ceph-mon[33646]: osdmap e517: 8 total, 8 up, 8 in 2024-03-24T12:21:35.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:35 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-100"}]: dispatch 2024-03-24T12:21:35.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:35 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-100"}]: dispatch 2024-03-24T12:21:36.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:36 smithi090 ceph-mon[33646]: pgmap v795: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:36.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:36 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-100"}]': finished 2024-03-24T12:21:36.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:36 smithi090 ceph-mon[33646]: osdmap e518: 8 total, 8 up, 8 in 2024-03-24T12:21:36.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:36 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-100", "mode": "writeback"}]: dispatch 2024-03-24T12:21:36.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:36 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-100", "mode": "writeback"}]: dispatch 2024-03-24T12:21:36.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:36 smithi006 ceph-mon[30108]: pgmap v795: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:36.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:36 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-100"}]': finished 2024-03-24T12:21:36.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:36 smithi006 ceph-mon[30108]: osdmap e518: 8 total, 8 up, 8 in 2024-03-24T12:21:36.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:36 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-100", "mode": "writeback"}]: dispatch 2024-03-24T12:21:36.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:36 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-100", "mode": "writeback"}]: dispatch 2024-03-24T12:21:36.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:36 smithi006 ceph-mon[41734]: pgmap v795: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:36.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:36 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-6", "overlaypool": "test-rados-api-smithi006-82061-100"}]': finished 2024-03-24T12:21:36.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:36 smithi006 ceph-mon[41734]: osdmap e518: 8 total, 8 up, 8 in 2024-03-24T12:21:36.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:36 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-100", "mode": "writeback"}]: dispatch 2024-03-24T12:21:36.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:36 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-100", "mode": "writeback"}]: dispatch 2024-03-24T12:21:37.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:37 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:21:37.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:37 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-100", "mode": "writeback"}]': finished 2024-03-24T12:21:37.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:37 smithi090 ceph-mon[33646]: osdmap e519: 8 total, 8 up, 8 in 2024-03-24T12:21:37.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:21:37.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:37 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:21:37.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:37 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:21:37.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:37 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-100", "mode": "writeback"}]': finished 2024-03-24T12:21:37.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:37 smithi006 ceph-mon[30108]: osdmap e519: 8 total, 8 up, 8 in 2024-03-24T12:21:37.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:21:37.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:37 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:21:37.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:37 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:21:37.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:37 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-100", "mode": "writeback"}]': finished 2024-03-24T12:21:37.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:37 smithi006 ceph-mon[41734]: osdmap e519: 8 total, 8 up, 8 in 2024-03-24T12:21:37.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:21:37.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:37 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:21:38.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:38 smithi090 ceph-mon[33646]: pgmap v798: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:38.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:21:38.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:38 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:21:38.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:38 smithi090 ceph-mon[33646]: osdmap e520: 8 total, 8 up, 8 in 2024-03-24T12:21:38.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:21:38.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:38 smithi006 ceph-mon[30108]: pgmap v798: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:38.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:21:38.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:38 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:21:38.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:38 smithi006 ceph-mon[30108]: osdmap e520: 8 total, 8 up, 8 in 2024-03-24T12:21:38.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:21:38.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:38 smithi006 ceph-mon[41734]: pgmap v798: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:38.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:21:38.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:38 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:21:38.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:38 smithi006 ceph-mon[41734]: osdmap e520: 8 total, 8 up, 8 in 2024-03-24T12:21:38.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:21:39.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:21:39.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:39 smithi090 ceph-mon[33646]: osdmap e521: 8 total, 8 up, 8 in 2024-03-24T12:21:39.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:39 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-24T12:21:39.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:39 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-24T12:21:39.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:21:39.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:39 smithi006 ceph-mon[30108]: osdmap e521: 8 total, 8 up, 8 in 2024-03-24T12:21:39.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-24T12:21:39.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:39 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-24T12:21:39.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:21:39.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:39 smithi006 ceph-mon[41734]: osdmap e521: 8 total, 8 up, 8 in 2024-03-24T12:21:39.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:39 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-24T12:21:39.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:39 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-24T12:21:40.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:40 smithi090 ceph-mon[33646]: pgmap v801: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:40.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:40 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:21:40.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:40 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-24T12:21:40.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-24T12:21:40.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:40 smithi090 ceph-mon[33646]: osdmap e522: 8 total, 8 up, 8 in 2024-03-24T12:21:40.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:40 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-24T12:21:40.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:40 smithi006 ceph-mon[30108]: pgmap v801: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:40.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:40 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:21:40.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:40 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-24T12:21:40.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-24T12:21:40.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:40 smithi006 ceph-mon[30108]: osdmap e522: 8 total, 8 up, 8 in 2024-03-24T12:21:40.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:40 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-24T12:21:40.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:40 smithi006 ceph-mon[41734]: pgmap v801: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:40.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:40 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:21:40.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:40 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-24T12:21:40.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-24T12:21:40.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:40 smithi006 ceph-mon[41734]: osdmap e522: 8 total, 8 up, 8 in 2024-03-24T12:21:40.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:40 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-24T12:21:41.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:41 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-03-24T12:21:41.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:41 smithi090 ceph-mon[33646]: osdmap e523: 8 total, 8 up, 8 in 2024-03-24T12:21:41.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:41.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:41 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:41.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:41 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-03-24T12:21:41.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:41 smithi006 ceph-mon[30108]: osdmap e523: 8 total, 8 up, 8 in 2024-03-24T12:21:41.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:41.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:41 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:41.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:41 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-03-24T12:21:41.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:41 smithi006 ceph-mon[41734]: osdmap e523: 8 total, 8 up, 8 in 2024-03-24T12:21:41.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:41.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:41 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:42.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:42 smithi006 ceph-mon[30108]: pgmap v804: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:21:42.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:42 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:21:42.861 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:42 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-100"}]: dispatch 2024-03-24T12:21:42.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:42 smithi006 ceph-mon[30108]: osdmap e524: 8 total, 8 up, 8 in 2024-03-24T12:21:42.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:42 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-100"}]: dispatch 2024-03-24T12:21:42.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:42 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-100"}]': finished 2024-03-24T12:21:42.862 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:42 smithi006 ceph-mon[30108]: osdmap e525: 8 total, 8 up, 8 in 2024-03-24T12:21:42.862 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:42 smithi006 ceph-mon[41734]: pgmap v804: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:21:42.862 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:42 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:21:42.862 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:42 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-100"}]: dispatch 2024-03-24T12:21:42.862 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:42 smithi006 ceph-mon[41734]: osdmap e524: 8 total, 8 up, 8 in 2024-03-24T12:21:42.862 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:42 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-100"}]: dispatch 2024-03-24T12:21:42.862 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:42 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-100"}]': finished 2024-03-24T12:21:42.862 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:42 smithi006 ceph-mon[41734]: osdmap e525: 8 total, 8 up, 8 in 2024-03-24T12:21:42.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:21:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:21:42.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:42 smithi090 ceph-mon[33646]: pgmap v804: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:21:42.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:42 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]': finished 2024-03-24T12:21:42.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:42 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-100"}]: dispatch 2024-03-24T12:21:42.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:42 smithi090 ceph-mon[33646]: osdmap e524: 8 total, 8 up, 8 in 2024-03-24T12:21:42.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:42 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-100"}]: dispatch 2024-03-24T12:21:42.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:42 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-100"}]': finished 2024-03-24T12:21:42.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:42 smithi090 ceph-mon[33646]: osdmap e525: 8 total, 8 up, 8 in 2024-03-24T12:21:43.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:43 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:21:43.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:43 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:21:43.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:43 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:21:44.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:44 smithi090 ceph-mon[33646]: pgmap v807: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-24T12:21:44.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:44 smithi090 ceph-mon[33646]: osdmap e526: 8 total, 8 up, 8 in 2024-03-24T12:21:44.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:44 smithi006 ceph-mon[30108]: pgmap v807: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-24T12:21:44.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:44 smithi006 ceph-mon[30108]: osdmap e526: 8 total, 8 up, 8 in 2024-03-24T12:21:44.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:44 smithi006 ceph-mon[41734]: pgmap v807: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-24T12:21:44.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:44 smithi006 ceph-mon[41734]: osdmap e526: 8 total, 8 up, 8 in 2024-03-24T12:21:45.576 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:21:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:21:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:21:45.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:45 smithi090 ceph-mon[33646]: osdmap e527: 8 total, 8 up, 8 in 2024-03-24T12:21:45.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:45.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:45.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:45 smithi006 ceph-mon[30108]: osdmap e527: 8 total, 8 up, 8 in 2024-03-24T12:21:45.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:45.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:45.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:45 smithi006 ceph-mon[41734]: osdmap e527: 8 total, 8 up, 8 in 2024-03-24T12:21:45.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:45.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:46.585 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:46 smithi006 ceph-mon[30108]: pgmap v810: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:21:46.586 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:46 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:21:46.586 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:46 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:21:46.586 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:46 smithi006 ceph-mon[30108]: osdmap e528: 8 total, 8 up, 8 in 2024-03-24T12:21:46.586 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:46 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:21:46.586 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:46 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:21:46.586 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:46 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:21:46.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:46 smithi090 ceph-mon[33646]: pgmap v810: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:21:46.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:46 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:21:46.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:46 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:21:46.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:46 smithi090 ceph-mon[33646]: osdmap e528: 8 total, 8 up, 8 in 2024-03-24T12:21:46.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:46 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:21:46.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:46 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:21:46.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:46 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:21:46.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:46 smithi006 ceph-mon[41734]: pgmap v810: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:21:46.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:46 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:21:46.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:46 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:21:46.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:46 smithi006 ceph-mon[41734]: osdmap e528: 8 total, 8 up, 8 in 2024-03-24T12:21:46.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:46 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-24T12:21:46.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:46 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:21:46.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:46 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-24T12:21:47.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:47 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:21:47.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:47 smithi090 ceph-mon[33646]: osdmap e529: 8 total, 8 up, 8 in 2024-03-24T12:21:47.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:21:47.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:47 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:21:47.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:47 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-24T12:21:47.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:47 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:21:47.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:47 smithi006 ceph-mon[30108]: osdmap e529: 8 total, 8 up, 8 in 2024-03-24T12:21:47.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:21:47.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:47 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:21:47.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:47 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-24T12:21:47.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:47 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-24T12:21:47.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:47 smithi006 ceph-mon[41734]: osdmap e529: 8 total, 8 up, 8 in 2024-03-24T12:21:47.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:21:47.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:47 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-24T12:21:47.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:47 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-24T12:21:48.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:48 smithi090 ceph-mon[33646]: pgmap v813: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:48.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:48 smithi090 ceph-mon[33646]: osdmap e530: 8 total, 8 up, 8 in 2024-03-24T12:21:48.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:21:48.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:48 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:21:48.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:48 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:21:48.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:48 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-24T12:21:48.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:48 smithi090 ceph-mon[33646]: osdmap e531: 8 total, 8 up, 8 in 2024-03-24T12:21:48.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:48 smithi006 ceph-mon[30108]: pgmap v813: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:48.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:48 smithi006 ceph-mon[30108]: osdmap e530: 8 total, 8 up, 8 in 2024-03-24T12:21:48.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:21:48.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:48 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:21:48.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:48 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:21:48.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:48 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-24T12:21:48.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:48 smithi006 ceph-mon[30108]: osdmap e531: 8 total, 8 up, 8 in 2024-03-24T12:21:48.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:48 smithi006 ceph-mon[41734]: pgmap v813: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:48.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:48 smithi006 ceph-mon[41734]: osdmap e530: 8 total, 8 up, 8 in 2024-03-24T12:21:48.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:21:48.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:48 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-24T12:21:48.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:48 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:21:48.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:48 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-24T12:21:48.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:48 smithi006 ceph-mon[41734]: osdmap e531: 8 total, 8 up, 8 in 2024-03-24T12:21:49.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:49 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:49.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:49 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:49.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:49 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-102"}]: dispatch 2024-03-24T12:21:49.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:49 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-102"}]: dispatch 2024-03-24T12:21:49.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:49 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:49 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:49 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-102"}]: dispatch 2024-03-24T12:21:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:49 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-102"}]: dispatch 2024-03-24T12:21:49.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:49 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:49.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:49 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-6"}]: dispatch 2024-03-24T12:21:49.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:49 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/297820617' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-102"}]: dispatch 2024-03-24T12:21:49.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:49 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-6", "tierpool": "test-rados-api-smithi006-82061-102"}]: dispatch 2024-03-24T12:21:50.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:50 smithi090 ceph-mon[33646]: pgmap v816: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:50.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:50 smithi090 ceph-mon[33646]: osdmap e532: 8 total, 8 up, 8 in 2024-03-24T12:21:50.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:50 smithi006 ceph-mon[30108]: pgmap v816: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:50.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:50 smithi006 ceph-mon[30108]: osdmap e532: 8 total, 8 up, 8 in 2024-03-24T12:21:50.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:50 smithi006 ceph-mon[41734]: pgmap v816: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:50.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:50 smithi006 ceph-mon[41734]: osdmap e532: 8 total, 8 up, 8 in 2024-03-24T12:21:51.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:51 smithi090 ceph-mon[33646]: osdmap e533: 8 total, 8 up, 8 in 2024-03-24T12:21:51.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:51 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi006-82061-104"}]: dispatch 2024-03-24T12:21:51.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:51 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi006-82061-104"}]: dispatch 2024-03-24T12:21:51.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:51 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi006-82061-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:21:51.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:51 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:21:51.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:51 smithi006 ceph-mon[30108]: osdmap e533: 8 total, 8 up, 8 in 2024-03-24T12:21:51.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi006-82061-104"}]: dispatch 2024-03-24T12:21:51.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi006-82061-104"}]: dispatch 2024-03-24T12:21:51.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi006-82061-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:21:51.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:51 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:21:51.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:51 smithi006 ceph-mon[41734]: osdmap e533: 8 total, 8 up, 8 in 2024-03-24T12:21:51.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:51 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi006-82061-104"}]: dispatch 2024-03-24T12:21:51.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:51 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi006-82061-104"}]: dispatch 2024-03-24T12:21:51.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:51 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi006-82061-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:21:51.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:51 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:21:52.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:21:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:21:52.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:52 smithi090 ceph-mon[33646]: pgmap v819: 228 pgs: 228 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:21:52.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:52 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi006-82061-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:21:52.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:52 smithi090 ceph-mon[33646]: osdmap e534: 8 total, 8 up, 8 in 2024-03-24T12:21:52.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:52 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi006-82061-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi006-82061-104"}]: dispatch 2024-03-24T12:21:52.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:52 smithi006 ceph-mon[30108]: pgmap v819: 228 pgs: 228 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:21:52.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:52 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi006-82061-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:21:52.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:52 smithi006 ceph-mon[30108]: osdmap e534: 8 total, 8 up, 8 in 2024-03-24T12:21:52.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:52 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi006-82061-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi006-82061-104"}]: dispatch 2024-03-24T12:21:52.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:52 smithi006 ceph-mon[41734]: pgmap v819: 228 pgs: 228 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:21:52.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:52 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi006-82061-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:21:52.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:52 smithi006 ceph-mon[41734]: osdmap e534: 8 total, 8 up, 8 in 2024-03-24T12:21:52.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:52 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi006-82061-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi006-82061-104"}]: dispatch 2024-03-24T12:21:53.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:53 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:21:53.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:53 smithi090 ceph-mon[33646]: osdmap e535: 8 total, 8 up, 8 in 2024-03-24T12:21:53.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:53 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:21:53.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:53 smithi006 ceph-mon[30108]: osdmap e535: 8 total, 8 up, 8 in 2024-03-24T12:21:53.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:53 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:21:53.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:53 smithi006 ceph-mon[41734]: osdmap e535: 8 total, 8 up, 8 in 2024-03-24T12:21:54.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:54 smithi090 ceph-mon[33646]: pgmap v822: 228 pgs: 228 active+clean; 582 KiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:21:54.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi006-82061-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi006-82061-104"}]': finished 2024-03-24T12:21:54.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:54 smithi090 ceph-mon[33646]: osdmap e536: 8 total, 8 up, 8 in 2024-03-24T12:21:54.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:54 smithi006 ceph-mon[30108]: pgmap v822: 228 pgs: 228 active+clean; 582 KiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:21:54.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi006-82061-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi006-82061-104"}]': finished 2024-03-24T12:21:54.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:54 smithi006 ceph-mon[30108]: osdmap e536: 8 total, 8 up, 8 in 2024-03-24T12:21:54.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:54 smithi006 ceph-mon[41734]: pgmap v822: 228 pgs: 228 active+clean; 582 KiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:21:54.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi006-82061-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi006-82061-104"}]': finished 2024-03-24T12:21:54.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:54 smithi006 ceph-mon[41734]: osdmap e536: 8 total, 8 up, 8 in 2024-03-24T12:21:55.644 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:21:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:21:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:21:55.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:55 smithi090 ceph-mon[33646]: osdmap e537: 8 total, 8 up, 8 in 2024-03-24T12:21:55.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:55 smithi006 ceph-mon[30108]: osdmap e537: 8 total, 8 up, 8 in 2024-03-24T12:21:55.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:55 smithi006 ceph-mon[41734]: osdmap e537: 8 total, 8 up, 8 in 2024-03-24T12:21:56.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:56 smithi090 ceph-mon[33646]: pgmap v825: 236 pgs: 8 unknown, 228 active+clean; 582 KiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:56.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:56 smithi090 ceph-mon[33646]: osdmap e538: 8 total, 8 up, 8 in 2024-03-24T12:21:56.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:56 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:56.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:56 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:21:56.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:56 smithi006 ceph-mon[30108]: pgmap v825: 236 pgs: 8 unknown, 228 active+clean; 582 KiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:56.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:56 smithi006 ceph-mon[30108]: osdmap e538: 8 total, 8 up, 8 in 2024-03-24T12:21:56.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:56 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:56.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:56 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:21:56.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:56 smithi006 ceph-mon[41734]: pgmap v825: 236 pgs: 8 unknown, 228 active+clean; 582 KiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:21:56.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:56 smithi006 ceph-mon[41734]: osdmap e538: 8 total, 8 up, 8 in 2024-03-24T12:21:56.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:56 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:56.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:56 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:21:57.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:57 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:21:57.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:57 smithi090 ceph-mon[33646]: osdmap e539: 8 total, 8 up, 8 in 2024-03-24T12:21:57.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:57 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:21:57.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:57 smithi006 ceph-mon[30108]: osdmap e539: 8 total, 8 up, 8 in 2024-03-24T12:21:57.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:57 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:21:57.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:57 smithi006 ceph-mon[41734]: osdmap e539: 8 total, 8 up, 8 in 2024-03-24T12:21:58.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:58 smithi090 ceph-mon[33646]: pgmap v828: 268 pgs: 268 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:21:58.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:58 smithi090 ceph-mon[33646]: osdmap e540: 8 total, 8 up, 8 in 2024-03-24T12:21:58.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:58 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:58.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:58 smithi006 ceph-mon[30108]: pgmap v828: 268 pgs: 268 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:21:58.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:58 smithi006 ceph-mon[30108]: osdmap e540: 8 total, 8 up, 8 in 2024-03-24T12:21:58.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:58 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:58.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:58 smithi006 ceph-mon[41734]: pgmap v828: 268 pgs: 268 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:21:58.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:58 smithi006 ceph-mon[41734]: osdmap e540: 8 total, 8 up, 8 in 2024-03-24T12:21:58.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:58 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:21:59.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:59 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:21:59.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:59 smithi090 ceph-mon[33646]: osdmap e541: 8 total, 8 up, 8 in 2024-03-24T12:21:59.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:59 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-107", "tierpool": "test-rados-api-smithi006-82061-107-cache"}]: dispatch 2024-03-24T12:21:59.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:59 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-107", "tierpool": "test-rados-api-smithi006-82061-107-cache"}]': finished 2024-03-24T12:21:59.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:21:59 smithi090 ceph-mon[33646]: osdmap e542: 8 total, 8 up, 8 in 2024-03-24T12:21:59.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:59 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:21:59.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:59 smithi006 ceph-mon[30108]: osdmap e541: 8 total, 8 up, 8 in 2024-03-24T12:21:59.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:59 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-107", "tierpool": "test-rados-api-smithi006-82061-107-cache"}]: dispatch 2024-03-24T12:21:59.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:59 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-107", "tierpool": "test-rados-api-smithi006-82061-107-cache"}]': finished 2024-03-24T12:21:59.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:21:59 smithi006 ceph-mon[30108]: osdmap e542: 8 total, 8 up, 8 in 2024-03-24T12:21:59.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:59 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:21:59.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:59 smithi006 ceph-mon[41734]: osdmap e541: 8 total, 8 up, 8 in 2024-03-24T12:21:59.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:59 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-107", "tierpool": "test-rados-api-smithi006-82061-107-cache"}]: dispatch 2024-03-24T12:21:59.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:59 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-107", "tierpool": "test-rados-api-smithi006-82061-107-cache"}]': finished 2024-03-24T12:21:59.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:21:59 smithi006 ceph-mon[41734]: osdmap e542: 8 total, 8 up, 8 in 2024-03-24T12:22:00.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:00 smithi006 ceph-mon[30108]: pgmap v831: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:22:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:00 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-107", "overlaypool": "test-rados-api-smithi006-82061-107-cache"}]: dispatch 2024-03-24T12:22:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:00 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:22:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:00 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-107", "overlaypool": "test-rados-api-smithi006-82061-107-cache"}]': finished 2024-03-24T12:22:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:00 smithi006 ceph-mon[30108]: osdmap e543: 8 total, 8 up, 8 in 2024-03-24T12:22:00.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:00 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-107-cache", "mode": "writeback"}]: dispatch 2024-03-24T12:22:00.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:00 smithi006 ceph-mon[41734]: pgmap v831: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:22:00.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:00 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-107", "overlaypool": "test-rados-api-smithi006-82061-107-cache"}]: dispatch 2024-03-24T12:22:00.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:00 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:22:00.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:00 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-107", "overlaypool": "test-rados-api-smithi006-82061-107-cache"}]': finished 2024-03-24T12:22:00.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:00 smithi006 ceph-mon[41734]: osdmap e543: 8 total, 8 up, 8 in 2024-03-24T12:22:00.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:00 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-107-cache", "mode": "writeback"}]: dispatch 2024-03-24T12:22:01.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:00 smithi090 ceph-mon[33646]: pgmap v831: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:22:01.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:00 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-107", "overlaypool": "test-rados-api-smithi006-82061-107-cache"}]: dispatch 2024-03-24T12:22:01.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:00 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:22:01.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:00 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-107", "overlaypool": "test-rados-api-smithi006-82061-107-cache"}]': finished 2024-03-24T12:22:01.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:00 smithi090 ceph-mon[33646]: osdmap e543: 8 total, 8 up, 8 in 2024-03-24T12:22:01.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:00 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-107-cache", "mode": "writeback"}]: dispatch 2024-03-24T12:22:01.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:01 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:22:01.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:01 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:22:01.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:01 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:22:01.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:01 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:22:01.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-107-cache", "mode": "writeback"}]': finished 2024-03-24T12:22:01.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:01 smithi006 ceph-mon[30108]: osdmap e544: 8 total, 8 up, 8 in 2024-03-24T12:22:01.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:01 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:22:01.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:01 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:22:01.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:01 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:22:01.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:01 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:22:01.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-107-cache", "mode": "writeback"}]': finished 2024-03-24T12:22:01.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:01 smithi006 ceph-mon[41734]: osdmap e544: 8 total, 8 up, 8 in 2024-03-24T12:22:02.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:01 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:22:02.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:01 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:22:02.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:01 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:22:02.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:01 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:22:02.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-107-cache", "mode": "writeback"}]': finished 2024-03-24T12:22:02.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:01 smithi090 ceph-mon[33646]: osdmap e544: 8 total, 8 up, 8 in 2024-03-24T12:22:02.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:02 smithi090 ceph-mon[33646]: pgmap v834: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:22:02.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:02 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-107"}]: dispatch 2024-03-24T12:22:02.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:22:02 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:22:02.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:02 smithi006 ceph-mon[30108]: pgmap v834: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:22:02.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:02 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-107"}]: dispatch 2024-03-24T12:22:02.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:02 smithi006 ceph-mon[41734]: pgmap v834: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:22:02.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:02 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-107"}]: dispatch 2024-03-24T12:22:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:03 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:22:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-107"}]': finished 2024-03-24T12:22:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:03 smithi006 ceph-mon[30108]: osdmap e545: 8 total, 8 up, 8 in 2024-03-24T12:22:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-107", "tierpool": "test-rados-api-smithi006-82061-107-cache"}]: dispatch 2024-03-24T12:22:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:03 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:22:03.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:03 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:22:03.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:03 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:22:03.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-107"}]': finished 2024-03-24T12:22:03.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:03 smithi006 ceph-mon[41734]: osdmap e545: 8 total, 8 up, 8 in 2024-03-24T12:22:03.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-107", "tierpool": "test-rados-api-smithi006-82061-107-cache"}]: dispatch 2024-03-24T12:22:03.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:03 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:22:03.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:03 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:22:04.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:03 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:22:04.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-107"}]': finished 2024-03-24T12:22:04.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:03 smithi090 ceph-mon[33646]: osdmap e545: 8 total, 8 up, 8 in 2024-03-24T12:22:04.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-107", "tierpool": "test-rados-api-smithi006-82061-107-cache"}]: dispatch 2024-03-24T12:22:04.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:03 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:22:04.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:03 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:22:04.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:04 smithi006 ceph-mon[30108]: pgmap v837: 300 pgs: 300 active+clean; 582 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-24T12:22:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:04 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:22:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:04 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-107", "tierpool": "test-rados-api-smithi006-82061-107-cache"}]': finished 2024-03-24T12:22:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:04 smithi006 ceph-mon[30108]: osdmap e546: 8 total, 8 up, 8 in 2024-03-24T12:22:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:04 smithi006 ceph-mon[41734]: pgmap v837: 300 pgs: 300 active+clean; 582 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-24T12:22:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:04 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:22:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:04 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-107", "tierpool": "test-rados-api-smithi006-82061-107-cache"}]': finished 2024-03-24T12:22:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:04 smithi006 ceph-mon[41734]: osdmap e546: 8 total, 8 up, 8 in 2024-03-24T12:22:05.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:04 smithi090 ceph-mon[33646]: pgmap v837: 300 pgs: 300 active+clean; 582 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-24T12:22:05.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:04 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:22:05.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:04 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/628528869' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-107", "tierpool": "test-rados-api-smithi006-82061-107-cache"}]': finished 2024-03-24T12:22:05.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:04 smithi090 ceph-mon[33646]: osdmap e546: 8 total, 8 up, 8 in 2024-03-24T12:22:05.706 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:22:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:22:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:22:05.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:05 smithi006 ceph-mon[30108]: osdmap e547: 8 total, 8 up, 8 in 2024-03-24T12:22:05.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:05 smithi006 ceph-mon[30108]: pgmap v840: 268 pgs: 268 active+clean; 582 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:22:05.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:05 smithi006 ceph-mon[41734]: osdmap e547: 8 total, 8 up, 8 in 2024-03-24T12:22:05.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:05 smithi006 ceph-mon[41734]: pgmap v840: 268 pgs: 268 active+clean; 582 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:22:06.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:05 smithi090 ceph-mon[33646]: osdmap e547: 8 total, 8 up, 8 in 2024-03-24T12:22:06.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:05 smithi090 ceph-mon[33646]: pgmap v840: 268 pgs: 268 active+clean; 582 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:22:07.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:06 smithi090 ceph-mon[33646]: osdmap e548: 8 total, 8 up, 8 in 2024-03-24T12:22:07.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:06 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:07.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:06 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:07.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:06 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:07.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:06 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:07.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:06 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82061-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:22:07.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:06 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82061-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:22:07.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:06 smithi006 ceph-mon[30108]: osdmap e548: 8 total, 8 up, 8 in 2024-03-24T12:22:07.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:06 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:07.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:06 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:07.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:06 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:07.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:06 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:07.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:06 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82061-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:22:07.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:06 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82061-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:22:07.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:06 smithi006 ceph-mon[41734]: osdmap e548: 8 total, 8 up, 8 in 2024-03-24T12:22:07.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:06 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:07.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:06 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:07.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:06 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:07.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:06 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:07.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:06 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82061-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:22:07.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:06 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82061-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:22:08.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:07 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82061-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:22:08.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:07 smithi090 ceph-mon[33646]: osdmap e549: 8 total, 8 up, 8 in 2024-03-24T12:22:08.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:07 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82061-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:08.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:07 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82061-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:08.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:07 smithi090 ceph-mon[33646]: pgmap v843: 236 pgs: 236 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:08.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:07 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82061-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:22:08.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:07 smithi006 ceph-mon[30108]: osdmap e549: 8 total, 8 up, 8 in 2024-03-24T12:22:08.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:07 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82061-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:08.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:07 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82061-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:08.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:07 smithi006 ceph-mon[30108]: pgmap v843: 236 pgs: 236 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:08.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:07 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82061-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:22:08.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:07 smithi006 ceph-mon[41734]: osdmap e549: 8 total, 8 up, 8 in 2024-03-24T12:22:08.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:07 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82061-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:08.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:07 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82061-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:08.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:07 smithi006 ceph-mon[41734]: pgmap v843: 236 pgs: 236 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:09.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:08 smithi090 ceph-mon[33646]: osdmap e550: 8 total, 8 up, 8 in 2024-03-24T12:22:09.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:08 smithi006 ceph-mon[30108]: osdmap e550: 8 total, 8 up, 8 in 2024-03-24T12:22:09.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:08 smithi006 ceph-mon[41734]: osdmap e550: 8 total, 8 up, 8 in 2024-03-24T12:22:10.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:09 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82061-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82061-109"}]': finished 2024-03-24T12:22:10.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:09 smithi090 ceph-mon[33646]: osdmap e551: 8 total, 8 up, 8 in 2024-03-24T12:22:10.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:09 smithi090 ceph-mon[33646]: pgmap v846: 244 pgs: 8 unknown, 236 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:10.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:09 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82061-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82061-109"}]': finished 2024-03-24T12:22:10.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:09 smithi006 ceph-mon[30108]: osdmap e551: 8 total, 8 up, 8 in 2024-03-24T12:22:10.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:09 smithi006 ceph-mon[30108]: pgmap v846: 244 pgs: 8 unknown, 236 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:10.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:09 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82061-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82061-109"}]': finished 2024-03-24T12:22:10.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:09 smithi006 ceph-mon[41734]: osdmap e551: 8 total, 8 up, 8 in 2024-03-24T12:22:10.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:09 smithi006 ceph-mon[41734]: pgmap v846: 244 pgs: 8 unknown, 236 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:10 smithi090 ceph-mon[33646]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:22:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:10 smithi090 ceph-mon[33646]: osdmap e552: 8 total, 8 up, 8 in 2024-03-24T12:22:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:22:11.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:10 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:22:11.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:10 smithi006 ceph-mon[30108]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:22:11.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:10 smithi006 ceph-mon[30108]: osdmap e552: 8 total, 8 up, 8 in 2024-03-24T12:22:11.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:22:11.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:10 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:22:11.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:10 smithi006 ceph-mon[41734]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:22:11.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:10 smithi006 ceph-mon[41734]: osdmap e552: 8 total, 8 up, 8 in 2024-03-24T12:22:11.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:22:11.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:10 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:22:12.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:22:12.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:11 smithi006 ceph-mon[30108]: osdmap e553: 8 total, 8 up, 8 in 2024-03-24T12:22:12.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-109", "tierpool": "test-rados-api-smithi006-82061-109-cache"}]: dispatch 2024-03-24T12:22:12.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:11 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-109", "tierpool": "test-rados-api-smithi006-82061-109-cache"}]: dispatch 2024-03-24T12:22:12.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:11 smithi006 ceph-mon[30108]: pgmap v849: 276 pgs: 40 unknown, 236 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:22:12.113 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:22:12.113 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:11 smithi006 ceph-mon[41734]: osdmap e553: 8 total, 8 up, 8 in 2024-03-24T12:22:12.113 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-109", "tierpool": "test-rados-api-smithi006-82061-109-cache"}]: dispatch 2024-03-24T12:22:12.113 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:11 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-109", "tierpool": "test-rados-api-smithi006-82061-109-cache"}]: dispatch 2024-03-24T12:22:12.113 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:11 smithi006 ceph-mon[41734]: pgmap v849: 276 pgs: 40 unknown, 236 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:22:12.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:22:12.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:11 smithi090 ceph-mon[33646]: osdmap e553: 8 total, 8 up, 8 in 2024-03-24T12:22:12.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-109", "tierpool": "test-rados-api-smithi006-82061-109-cache"}]: dispatch 2024-03-24T12:22:12.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:11 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-109", "tierpool": "test-rados-api-smithi006-82061-109-cache"}]: dispatch 2024-03-24T12:22:12.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:11 smithi090 ceph-mon[33646]: pgmap v849: 276 pgs: 40 unknown, 236 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:22:12.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:22:12 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:22:12.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-109", "tierpool": "test-rados-api-smithi006-82061-109-cache"}]': finished 2024-03-24T12:22:12.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:12 smithi090 ceph-mon[33646]: osdmap e554: 8 total, 8 up, 8 in 2024-03-24T12:22:12.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:12 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-109", "overlaypool": "test-rados-api-smithi006-82061-109-cache"}]: dispatch 2024-03-24T12:22:12.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:12 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-109", "overlaypool": "test-rados-api-smithi006-82061-109-cache"}]: dispatch 2024-03-24T12:22:12.929 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:12 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:22:13.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-109", "tierpool": "test-rados-api-smithi006-82061-109-cache"}]': finished 2024-03-24T12:22:13.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:12 smithi006 ceph-mon[30108]: osdmap e554: 8 total, 8 up, 8 in 2024-03-24T12:22:13.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:12 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-109", "overlaypool": "test-rados-api-smithi006-82061-109-cache"}]: dispatch 2024-03-24T12:22:13.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:12 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-109", "overlaypool": "test-rados-api-smithi006-82061-109-cache"}]: dispatch 2024-03-24T12:22:13.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:12 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:22:13.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-109", "tierpool": "test-rados-api-smithi006-82061-109-cache"}]': finished 2024-03-24T12:22:13.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:12 smithi006 ceph-mon[41734]: osdmap e554: 8 total, 8 up, 8 in 2024-03-24T12:22:13.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:12 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-109", "overlaypool": "test-rados-api-smithi006-82061-109-cache"}]: dispatch 2024-03-24T12:22:13.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:12 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-109", "overlaypool": "test-rados-api-smithi006-82061-109-cache"}]: dispatch 2024-03-24T12:22:13.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:12 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:22:14.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:13 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-109", "overlaypool": "test-rados-api-smithi006-82061-109-cache"}]': finished 2024-03-24T12:22:14.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:13 smithi090 ceph-mon[33646]: osdmap e555: 8 total, 8 up, 8 in 2024-03-24T12:22:14.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:13 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-109-cache", "mode": "writeback"}]: dispatch 2024-03-24T12:22:14.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:13 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-109-cache", "mode": "writeback"}]: dispatch 2024-03-24T12:22:14.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:13 smithi090 ceph-mon[33646]: pgmap v852: 276 pgs: 276 active+clean; 582 KiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:14.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:13 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:22:14.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:13 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-109-cache", "mode": "writeback"}]': finished 2024-03-24T12:22:14.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:13 smithi090 ceph-mon[33646]: osdmap e556: 8 total, 8 up, 8 in 2024-03-24T12:22:14.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:13 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-109", "overlaypool": "test-rados-api-smithi006-82061-109-cache"}]': finished 2024-03-24T12:22:14.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:13 smithi006 ceph-mon[30108]: osdmap e555: 8 total, 8 up, 8 in 2024-03-24T12:22:14.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:13 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-109-cache", "mode": "writeback"}]: dispatch 2024-03-24T12:22:14.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:13 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-109-cache", "mode": "writeback"}]: dispatch 2024-03-24T12:22:14.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:13 smithi006 ceph-mon[30108]: pgmap v852: 276 pgs: 276 active+clean; 582 KiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:14.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:13 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:22:14.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:13 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-109-cache", "mode": "writeback"}]': finished 2024-03-24T12:22:14.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:13 smithi006 ceph-mon[30108]: osdmap e556: 8 total, 8 up, 8 in 2024-03-24T12:22:14.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:13 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-109", "overlaypool": "test-rados-api-smithi006-82061-109-cache"}]': finished 2024-03-24T12:22:14.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:13 smithi006 ceph-mon[41734]: osdmap e555: 8 total, 8 up, 8 in 2024-03-24T12:22:14.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:13 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-109-cache", "mode": "writeback"}]: dispatch 2024-03-24T12:22:14.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:13 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-109-cache", "mode": "writeback"}]: dispatch 2024-03-24T12:22:14.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:13 smithi006 ceph-mon[41734]: pgmap v852: 276 pgs: 276 active+clean; 582 KiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:14.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:13 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:22:14.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:13 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-109-cache", "mode": "writeback"}]': finished 2024-03-24T12:22:14.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:13 smithi006 ceph-mon[41734]: osdmap e556: 8 total, 8 up, 8 in 2024-03-24T12:22:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:22:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:14 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:22:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:14 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:22:15.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:14 smithi090 ceph-mon[33646]: osdmap e557: 8 total, 8 up, 8 in 2024-03-24T12:22:15.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:22:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:14 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:22:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:14 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:22:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:14 smithi006 ceph-mon[30108]: osdmap e557: 8 total, 8 up, 8 in 2024-03-24T12:22:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:22:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:14 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:22:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:14 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:22:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:14 smithi006 ceph-mon[41734]: osdmap e557: 8 total, 8 up, 8 in 2024-03-24T12:22:15.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:22:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:22:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:22:16.565 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:22:16.565 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:15 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:22:16.565 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:15 smithi090 ceph-mon[33646]: pgmap v855: 276 pgs: 276 active+clean; 582 KiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:16.566 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:22:16.566 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:15 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:22:16.566 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:15 smithi006 ceph-mon[30108]: pgmap v855: 276 pgs: 276 active+clean; 582 KiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:16.566 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:22:16.566 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:15 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:22:16.566 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:15 smithi006 ceph-mon[41734]: pgmap v855: 276 pgs: 276 active+clean; 582 KiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:17.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:16 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:22:17.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:16 smithi090 ceph-mon[33646]: osdmap e558: 8 total, 8 up, 8 in 2024-03-24T12:22:17.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:16 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-24T12:22:17.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:16 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-24T12:22:17.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:16 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:22:17.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:16 smithi006 ceph-mon[41734]: osdmap e558: 8 total, 8 up, 8 in 2024-03-24T12:22:17.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:16 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-24T12:22:17.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:16 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-24T12:22:17.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:16 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:22:17.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:16 smithi006 ceph-mon[30108]: osdmap e558: 8 total, 8 up, 8 in 2024-03-24T12:22:17.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:16 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-24T12:22:17.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:16 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-24T12:22:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:17 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:22:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-24T12:22:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:17 smithi090 ceph-mon[33646]: osdmap e559: 8 total, 8 up, 8 in 2024-03-24T12:22:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-24T12:22:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-24T12:22:18.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:17 smithi090 ceph-mon[33646]: pgmap v858: 276 pgs: 276 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:18.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:17 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:22:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-24T12:22:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:17 smithi006 ceph-mon[30108]: osdmap e559: 8 total, 8 up, 8 in 2024-03-24T12:22:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-24T12:22:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-24T12:22:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:17 smithi006 ceph-mon[30108]: pgmap v858: 276 pgs: 276 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:18.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:17 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:22:18.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-24T12:22:18.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:17 smithi006 ceph-mon[41734]: osdmap e559: 8 total, 8 up, 8 in 2024-03-24T12:22:18.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-24T12:22:18.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-24T12:22:18.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:17 smithi006 ceph-mon[41734]: pgmap v858: 276 pgs: 276 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:19.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-03-24T12:22:19.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:18 smithi090 ceph-mon[33646]: osdmap e560: 8 total, 8 up, 8 in 2024-03-24T12:22:19.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:19.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:19.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:18 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:22:19.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:18 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:22:19.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-03-24T12:22:19.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:18 smithi006 ceph-mon[30108]: osdmap e560: 8 total, 8 up, 8 in 2024-03-24T12:22:19.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:19.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:19.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:18 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:22:19.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:18 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:22:19.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-03-24T12:22:19.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:18 smithi006 ceph-mon[41734]: osdmap e560: 8 total, 8 up, 8 in 2024-03-24T12:22:19.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:19.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:19.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:18 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:22:19.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:18 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:22:20.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-109"}]': finished 2024-03-24T12:22:20.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:19 smithi090 ceph-mon[33646]: osdmap e561: 8 total, 8 up, 8 in 2024-03-24T12:22:20.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-109", "tierpool": "test-rados-api-smithi006-82061-109-cache"}]: dispatch 2024-03-24T12:22:20.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-109", "tierpool": "test-rados-api-smithi006-82061-109-cache"}]: dispatch 2024-03-24T12:22:20.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:19 smithi090 ceph-mon[33646]: pgmap v861: 276 pgs: 276 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:20.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-109"}]': finished 2024-03-24T12:22:20.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:19 smithi006 ceph-mon[30108]: osdmap e561: 8 total, 8 up, 8 in 2024-03-24T12:22:20.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-109", "tierpool": "test-rados-api-smithi006-82061-109-cache"}]: dispatch 2024-03-24T12:22:20.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-109", "tierpool": "test-rados-api-smithi006-82061-109-cache"}]: dispatch 2024-03-24T12:22:20.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:19 smithi006 ceph-mon[30108]: pgmap v861: 276 pgs: 276 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:20.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-109"}]': finished 2024-03-24T12:22:20.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:19 smithi006 ceph-mon[41734]: osdmap e561: 8 total, 8 up, 8 in 2024-03-24T12:22:20.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-109", "tierpool": "test-rados-api-smithi006-82061-109-cache"}]: dispatch 2024-03-24T12:22:20.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-109", "tierpool": "test-rados-api-smithi006-82061-109-cache"}]: dispatch 2024-03-24T12:22:20.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:19 smithi006 ceph-mon[41734]: pgmap v861: 276 pgs: 276 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:21.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:20 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-109", "tierpool": "test-rados-api-smithi006-82061-109-cache"}]': finished 2024-03-24T12:22:21.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:20 smithi090 ceph-mon[33646]: osdmap e562: 8 total, 8 up, 8 in 2024-03-24T12:22:21.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:20 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-109", "tierpool": "test-rados-api-smithi006-82061-109-cache"}]': finished 2024-03-24T12:22:21.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:20 smithi006 ceph-mon[30108]: osdmap e562: 8 total, 8 up, 8 in 2024-03-24T12:22:21.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:20 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-109", "tierpool": "test-rados-api-smithi006-82061-109-cache"}]': finished 2024-03-24T12:22:21.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:20 smithi006 ceph-mon[41734]: osdmap e562: 8 total, 8 up, 8 in 2024-03-24T12:22:22.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:21 smithi090 ceph-mon[33646]: osdmap e563: 8 total, 8 up, 8 in 2024-03-24T12:22:22.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:21 smithi090 ceph-mon[33646]: pgmap v864: 244 pgs: 244 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:22:22.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:21 smithi006 ceph-mon[30108]: osdmap e563: 8 total, 8 up, 8 in 2024-03-24T12:22:22.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:21 smithi006 ceph-mon[30108]: pgmap v864: 244 pgs: 244 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:22:22.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:21 smithi006 ceph-mon[41734]: osdmap e563: 8 total, 8 up, 8 in 2024-03-24T12:22:22.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:21 smithi006 ceph-mon[41734]: pgmap v864: 244 pgs: 244 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:22:23.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:22 smithi090 ceph-mon[33646]: osdmap e564: 8 total, 8 up, 8 in 2024-03-24T12:22:23.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:23.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:23.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:22 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:22:23.179 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:22:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:22:23.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:22 smithi006 ceph-mon[30108]: osdmap e564: 8 total, 8 up, 8 in 2024-03-24T12:22:23.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:23.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:23.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:22 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:22:23.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:22 smithi006 ceph-mon[41734]: osdmap e564: 8 total, 8 up, 8 in 2024-03-24T12:22:23.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:23.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:23.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:22 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:22:24.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-109"}]': finished 2024-03-24T12:22:24.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:23 smithi090 ceph-mon[33646]: osdmap e565: 8 total, 8 up, 8 in 2024-03-24T12:22:24.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:24.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:24.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:23 smithi090 ceph-mon[33646]: pgmap v867: 236 pgs: 236 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:24.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-109"}]': finished 2024-03-24T12:22:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:23 smithi006 ceph-mon[30108]: osdmap e565: 8 total, 8 up, 8 in 2024-03-24T12:22:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:23 smithi006 ceph-mon[30108]: pgmap v867: 236 pgs: 236 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:24.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-109"}]': finished 2024-03-24T12:22:24.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:23 smithi006 ceph-mon[41734]: osdmap e565: 8 total, 8 up, 8 in 2024-03-24T12:22:24.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3266313414' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:24.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-109"}]: dispatch 2024-03-24T12:22:24.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:23 smithi006 ceph-mon[41734]: pgmap v867: 236 pgs: 236 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:25.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:24 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:22:25.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-109"}]': finished 2024-03-24T12:22:25.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:24 smithi090 ceph-mon[33646]: osdmap e566: 8 total, 8 up, 8 in 2024-03-24T12:22:25.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:24 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:22:25.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-109"}]': finished 2024-03-24T12:22:25.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:24 smithi006 ceph-mon[30108]: osdmap e566: 8 total, 8 up, 8 in 2024-03-24T12:22:25.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:24 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:22:25.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-109"}]': finished 2024-03-24T12:22:25.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:24 smithi006 ceph-mon[41734]: osdmap e566: 8 total, 8 up, 8 in 2024-03-24T12:22:25.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:22:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:22:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:22:26.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:25 smithi090 ceph-mon[33646]: osdmap e567: 8 total, 8 up, 8 in 2024-03-24T12:22:26.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi006-82061-104"}]: dispatch 2024-03-24T12:22:26.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:25 smithi090 ceph-mon[33646]: pgmap v870: 228 pgs: 228 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-24T12:22:26.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:25 smithi006 ceph-mon[30108]: osdmap e567: 8 total, 8 up, 8 in 2024-03-24T12:22:26.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi006-82061-104"}]: dispatch 2024-03-24T12:22:26.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:25 smithi006 ceph-mon[30108]: pgmap v870: 228 pgs: 228 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-24T12:22:26.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:25 smithi006 ceph-mon[41734]: osdmap e567: 8 total, 8 up, 8 in 2024-03-24T12:22:26.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi006-82061-104"}]: dispatch 2024-03-24T12:22:26.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:25 smithi006 ceph-mon[41734]: pgmap v870: 228 pgs: 228 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-24T12:22:27.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi006-82061-104"}]': finished 2024-03-24T12:22:27.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:26 smithi090 ceph-mon[33646]: osdmap e568: 8 total, 8 up, 8 in 2024-03-24T12:22:27.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi006-82061-104"}]: dispatch 2024-03-24T12:22:27.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi006-82061-104"}]': finished 2024-03-24T12:22:27.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:26 smithi006 ceph-mon[41734]: osdmap e568: 8 total, 8 up, 8 in 2024-03-24T12:22:27.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi006-82061-104"}]: dispatch 2024-03-24T12:22:27.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi006-82061-104"}]': finished 2024-03-24T12:22:27.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:26 smithi006 ceph-mon[30108]: osdmap e568: 8 total, 8 up, 8 in 2024-03-24T12:22:27.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi006-82061-104"}]: dispatch 2024-03-24T12:22:28.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:27 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi006-82061-104"}]': finished 2024-03-24T12:22:28.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:27 smithi090 ceph-mon[33646]: osdmap e569: 8 total, 8 up, 8 in 2024-03-24T12:22:28.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:27 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:22:28.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:27 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:22:28.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:27 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82061-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:22:28.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:27 smithi090 ceph-mon[33646]: pgmap v873: 228 pgs: 228 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-24T12:22:28.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:27 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi006-82061-104"}]': finished 2024-03-24T12:22:28.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:27 smithi006 ceph-mon[41734]: osdmap e569: 8 total, 8 up, 8 in 2024-03-24T12:22:28.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:27 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:22:28.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:27 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:22:28.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:27 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82061-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:22:28.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:27 smithi006 ceph-mon[41734]: pgmap v873: 228 pgs: 228 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-24T12:22:28.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:27 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1409382181' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi006-82061-104"}]': finished 2024-03-24T12:22:28.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:27 smithi006 ceph-mon[30108]: osdmap e569: 8 total, 8 up, 8 in 2024-03-24T12:22:28.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:27 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:22:28.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:27 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:22:28.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:27 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82061-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-24T12:22:28.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:27 smithi006 ceph-mon[30108]: pgmap v873: 228 pgs: 228 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-24T12:22:29.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82061-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:22:29.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:28 smithi090 ceph-mon[33646]: osdmap e570: 8 total, 8 up, 8 in 2024-03-24T12:22:29.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82061-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:22:29.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82061-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:22:29.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:28 smithi006 ceph-mon[30108]: osdmap e570: 8 total, 8 up, 8 in 2024-03-24T12:22:29.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82061-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:22:29.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi006-82061-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-24T12:22:29.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:28 smithi006 ceph-mon[41734]: osdmap e570: 8 total, 8 up, 8 in 2024-03-24T12:22:29.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82061-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:22:30.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:29 smithi090 ceph-mon[33646]: osdmap e571: 8 total, 8 up, 8 in 2024-03-24T12:22:30.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:29 smithi090 ceph-mon[33646]: pgmap v876: 228 pgs: 228 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:30.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:29 smithi006 ceph-mon[30108]: osdmap e571: 8 total, 8 up, 8 in 2024-03-24T12:22:30.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:29 smithi006 ceph-mon[30108]: pgmap v876: 228 pgs: 228 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:30.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:29 smithi006 ceph-mon[41734]: osdmap e571: 8 total, 8 up, 8 in 2024-03-24T12:22:30.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:29 smithi006 ceph-mon[41734]: pgmap v876: 228 pgs: 228 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:22:31.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:30 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82061-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:22:31.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:30 smithi090 ceph-mon[33646]: osdmap e572: 8 total, 8 up, 8 in 2024-03-24T12:22:31.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:30 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82061-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:22:31.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:30 smithi006 ceph-mon[30108]: osdmap e572: 8 total, 8 up, 8 in 2024-03-24T12:22:31.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:30 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi006-82061-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:22:31.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:30 smithi006 ceph-mon[41734]: osdmap e572: 8 total, 8 up, 8 in 2024-03-24T12:22:32.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:31 smithi090 ceph-mon[33646]: osdmap e573: 8 total, 8 up, 8 in 2024-03-24T12:22:32.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:22:32.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:31 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:22:32.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:31 smithi090 ceph-mon[33646]: pgmap v879: 268 pgs: 40 unknown, 228 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:22:32.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:22:32.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:31 smithi090 ceph-mon[33646]: osdmap e574: 8 total, 8 up, 8 in 2024-03-24T12:22:32.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:31 smithi006 ceph-mon[30108]: osdmap e573: 8 total, 8 up, 8 in 2024-03-24T12:22:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:22:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:31 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:22:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:31 smithi006 ceph-mon[30108]: pgmap v879: 268 pgs: 40 unknown, 228 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:22:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:22:32.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:31 smithi006 ceph-mon[30108]: osdmap e574: 8 total, 8 up, 8 in 2024-03-24T12:22:32.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:31 smithi006 ceph-mon[41734]: osdmap e573: 8 total, 8 up, 8 in 2024-03-24T12:22:32.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:22:32.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:31 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:22:32.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:31 smithi006 ceph-mon[41734]: pgmap v879: 268 pgs: 40 unknown, 228 active+clean; 582 KiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:22:32.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:22:32.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:31 smithi006 ceph-mon[41734]: osdmap e574: 8 total, 8 up, 8 in 2024-03-24T12:22:33.178 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:22:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:22:33.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:32 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:22:33.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:32 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:22:33.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:32 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-112", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:22:33.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:32 smithi090 ceph-mon[33646]: osdmap e575: 8 total, 8 up, 8 in 2024-03-24T12:22:33.179 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:32 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-112"}]: dispatch 2024-03-24T12:22:33.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:32 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:22:33.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:32 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:22:33.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:32 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-112", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:22:33.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:32 smithi006 ceph-mon[30108]: osdmap e575: 8 total, 8 up, 8 in 2024-03-24T12:22:33.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:32 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-112"}]: dispatch 2024-03-24T12:22:33.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:32 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:22:33.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:32 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:22:33.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:32 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-112", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:22:33.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:32 smithi006 ceph-mon[41734]: osdmap e575: 8 total, 8 up, 8 in 2024-03-24T12:22:33.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:32 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-112"}]: dispatch 2024-03-24T12:22:34.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:33 smithi090 ceph-mon[33646]: pgmap v882: 268 pgs: 268 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:22:34.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:33 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:22:34.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-112"}]': finished 2024-03-24T12:22:34.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:33 smithi090 ceph-mon[33646]: osdmap e576: 8 total, 8 up, 8 in 2024-03-24T12:22:34.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:33 smithi006 ceph-mon[30108]: pgmap v882: 268 pgs: 268 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:22:34.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:33 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:22:34.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-112"}]': finished 2024-03-24T12:22:34.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:33 smithi006 ceph-mon[30108]: osdmap e576: 8 total, 8 up, 8 in 2024-03-24T12:22:34.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:33 smithi006 ceph-mon[41734]: pgmap v882: 268 pgs: 268 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:22:34.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:33 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:22:34.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-112"}]': finished 2024-03-24T12:22:34.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:33 smithi006 ceph-mon[41734]: osdmap e576: 8 total, 8 up, 8 in 2024-03-24T12:22:35.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:22:35.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:22:35.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:22:35.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:22:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:22:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:22:36.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:35 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:22:36.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:35 smithi006 ceph-mon[30108]: osdmap e577: 8 total, 8 up, 8 in 2024-03-24T12:22:36.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:35 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-112"}]: dispatch 2024-03-24T12:22:36.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:35 smithi006 ceph-mon[30108]: pgmap v885: 268 pgs: 268 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:22:36.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:35 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:22:36.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:35 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:22:36.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:35 smithi006 ceph-mon[41734]: osdmap e577: 8 total, 8 up, 8 in 2024-03-24T12:22:36.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:35 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-112"}]: dispatch 2024-03-24T12:22:36.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:35 smithi006 ceph-mon[41734]: pgmap v885: 268 pgs: 268 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:22:36.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:35 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:22:36.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:35 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:22:36.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:35 smithi090 ceph-mon[33646]: osdmap e577: 8 total, 8 up, 8 in 2024-03-24T12:22:36.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:35 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-112"}]: dispatch 2024-03-24T12:22:36.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:35 smithi090 ceph-mon[33646]: pgmap v885: 268 pgs: 268 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:22:36.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:35 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:22:37.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-112"}]': finished 2024-03-24T12:22:37.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:37 smithi006 ceph-mon[30108]: osdmap e578: 8 total, 8 up, 8 in 2024-03-24T12:22:37.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-112"}]': finished 2024-03-24T12:22:37.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:37 smithi006 ceph-mon[41734]: osdmap e578: 8 total, 8 up, 8 in 2024-03-24T12:22:37.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-112"}]': finished 2024-03-24T12:22:37.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:37 smithi090 ceph-mon[33646]: osdmap e578: 8 total, 8 up, 8 in 2024-03-24T12:22:38.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:38 smithi006 ceph-mon[30108]: osdmap e579: 8 total, 8 up, 8 in 2024-03-24T12:22:38.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:38 smithi006 ceph-mon[30108]: pgmap v888: 236 pgs: 236 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-03-24T12:22:38.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:38 smithi006 ceph-mon[30108]: osdmap e580: 8 total, 8 up, 8 in 2024-03-24T12:22:38.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:38 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:22:38.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:38 smithi006 ceph-mon[41734]: osdmap e579: 8 total, 8 up, 8 in 2024-03-24T12:22:38.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:38 smithi006 ceph-mon[41734]: pgmap v888: 236 pgs: 236 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-03-24T12:22:38.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:38 smithi006 ceph-mon[41734]: osdmap e580: 8 total, 8 up, 8 in 2024-03-24T12:22:38.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:38 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:22:38.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:38 smithi090 ceph-mon[33646]: osdmap e579: 8 total, 8 up, 8 in 2024-03-24T12:22:38.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:38 smithi090 ceph-mon[33646]: pgmap v888: 236 pgs: 236 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-03-24T12:22:38.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:38 smithi090 ceph-mon[33646]: osdmap e580: 8 total, 8 up, 8 in 2024-03-24T12:22:38.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:38 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:22:40.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:22:40.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:39 smithi006 ceph-mon[30108]: osdmap e581: 8 total, 8 up, 8 in 2024-03-24T12:22:40.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:22:40.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:39 smithi006 ceph-mon[30108]: pgmap v891: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-03-24T12:22:40.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:39 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:22:40.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:39 smithi006 ceph-mon[41734]: osdmap e581: 8 total, 8 up, 8 in 2024-03-24T12:22:40.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:39 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:22:40.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:39 smithi006 ceph-mon[41734]: pgmap v891: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-03-24T12:22:40.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:39 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:22:40.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:39 smithi090 ceph-mon[33646]: osdmap e581: 8 total, 8 up, 8 in 2024-03-24T12:22:40.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:39 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:22:40.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:39 smithi090 ceph-mon[33646]: pgmap v891: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-03-24T12:22:41.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-114", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:22:41.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:40 smithi090 ceph-mon[33646]: osdmap e582: 8 total, 8 up, 8 in 2024-03-24T12:22:41.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-114"}]: dispatch 2024-03-24T12:22:41.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-114", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:22:41.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:40 smithi006 ceph-mon[30108]: osdmap e582: 8 total, 8 up, 8 in 2024-03-24T12:22:41.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-114"}]: dispatch 2024-03-24T12:22:41.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-114", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:22:41.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:40 smithi006 ceph-mon[41734]: osdmap e582: 8 total, 8 up, 8 in 2024-03-24T12:22:41.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-114"}]: dispatch 2024-03-24T12:22:42.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-114"}]': finished 2024-03-24T12:22:42.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:41 smithi006 ceph-mon[30108]: osdmap e583: 8 total, 8 up, 8 in 2024-03-24T12:22:42.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-114", "mode": "writeback"}]: dispatch 2024-03-24T12:22:42.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:41 smithi006 ceph-mon[30108]: pgmap v894: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:22:42.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:41 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:22:42.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-114", "mode": "writeback"}]': finished 2024-03-24T12:22:42.363 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:41 smithi006 ceph-mon[30108]: osdmap e584: 8 total, 8 up, 8 in 2024-03-24T12:22:42.363 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-114"}]': finished 2024-03-24T12:22:42.363 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:41 smithi006 ceph-mon[41734]: osdmap e583: 8 total, 8 up, 8 in 2024-03-24T12:22:42.363 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-114", "mode": "writeback"}]: dispatch 2024-03-24T12:22:42.363 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:41 smithi006 ceph-mon[41734]: pgmap v894: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:22:42.363 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:41 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:22:42.364 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-114", "mode": "writeback"}]': finished 2024-03-24T12:22:42.364 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:41 smithi006 ceph-mon[41734]: osdmap e584: 8 total, 8 up, 8 in 2024-03-24T12:22:42.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-114"}]': finished 2024-03-24T12:22:42.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:41 smithi090 ceph-mon[33646]: osdmap e583: 8 total, 8 up, 8 in 2024-03-24T12:22:42.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-114", "mode": "writeback"}]: dispatch 2024-03-24T12:22:42.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:41 smithi090 ceph-mon[33646]: pgmap v894: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:22:42.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:41 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:22:42.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-114", "mode": "writeback"}]': finished 2024-03-24T12:22:42.429 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:41 smithi090 ceph-mon[33646]: osdmap e584: 8 total, 8 up, 8 in 2024-03-24T12:22:42.978 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:22:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:22:43.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:42 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:22:43.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:42 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:22:43.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:42 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:22:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:43 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:22:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:43 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:22:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:43 smithi090 ceph-mon[33646]: osdmap e585: 8 total, 8 up, 8 in 2024-03-24T12:22:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:43 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-114"}]: dispatch 2024-03-24T12:22:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:43 smithi090 ceph-mon[33646]: pgmap v897: 268 pgs: 268 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-24T12:22:44.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:43 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:22:44.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:43 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:22:44.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:43 smithi006 ceph-mon[30108]: osdmap e585: 8 total, 8 up, 8 in 2024-03-24T12:22:44.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:43 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-114"}]: dispatch 2024-03-24T12:22:44.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:43 smithi006 ceph-mon[30108]: pgmap v897: 268 pgs: 268 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-24T12:22:44.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:43 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:22:44.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:43 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:22:44.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:43 smithi006 ceph-mon[41734]: osdmap e585: 8 total, 8 up, 8 in 2024-03-24T12:22:44.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:43 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-114"}]: dispatch 2024-03-24T12:22:44.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:43 smithi006 ceph-mon[41734]: pgmap v897: 268 pgs: 268 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-24T12:22:45.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:45 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:22:45.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-114"}]': finished 2024-03-24T12:22:45.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:45 smithi090 ceph-mon[33646]: osdmap e586: 8 total, 8 up, 8 in 2024-03-24T12:22:45.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:22:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:22:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:22:45.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:45 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:22:45.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-114"}]': finished 2024-03-24T12:22:45.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:45 smithi006 ceph-mon[30108]: osdmap e586: 8 total, 8 up, 8 in 2024-03-24T12:22:45.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:45 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:22:45.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-114"}]': finished 2024-03-24T12:22:45.713 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:45 smithi006 ceph-mon[41734]: osdmap e586: 8 total, 8 up, 8 in 2024-03-24T12:22:46.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:46 smithi090 ceph-mon[33646]: osdmap e587: 8 total, 8 up, 8 in 2024-03-24T12:22:47.029 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:46 smithi090 ceph-mon[33646]: pgmap v900: 236 pgs: 236 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:22:47.029 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:46 smithi090 ceph-mon[33646]: osdmap e588: 8 total, 8 up, 8 in 2024-03-24T12:22:47.029 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:46 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:22:47.030 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:46 smithi006 ceph-mon[30108]: osdmap e587: 8 total, 8 up, 8 in 2024-03-24T12:22:47.030 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:46 smithi006 ceph-mon[30108]: pgmap v900: 236 pgs: 236 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:22:47.030 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:46 smithi006 ceph-mon[30108]: osdmap e588: 8 total, 8 up, 8 in 2024-03-24T12:22:47.030 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:46 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:22:47.030 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:46 smithi006 ceph-mon[41734]: osdmap e587: 8 total, 8 up, 8 in 2024-03-24T12:22:47.031 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:46 smithi006 ceph-mon[41734]: pgmap v900: 236 pgs: 236 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:22:47.031 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:46 smithi006 ceph-mon[41734]: osdmap e588: 8 total, 8 up, 8 in 2024-03-24T12:22:47.031 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:46 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:22:48.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:22:48.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:48 smithi090 ceph-mon[33646]: osdmap e589: 8 total, 8 up, 8 in 2024-03-24T12:22:48.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:48 smithi090 ceph-mon[33646]: pgmap v903: 268 pgs: 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-03-24T12:22:48.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:22:48.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:48 smithi006 ceph-mon[30108]: osdmap e589: 8 total, 8 up, 8 in 2024-03-24T12:22:48.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:48 smithi006 ceph-mon[30108]: pgmap v903: 268 pgs: 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-03-24T12:22:48.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:22:48.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:48 smithi006 ceph-mon[41734]: osdmap e589: 8 total, 8 up, 8 in 2024-03-24T12:22:48.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:48 smithi006 ceph-mon[41734]: pgmap v903: 268 pgs: 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 1 op/s 2024-03-24T12:22:49.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:49 smithi090 ceph-mon[33646]: osdmap e590: 8 total, 8 up, 8 in 2024-03-24T12:22:49.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:49 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:22:49.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:49 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:22:49.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:49 smithi006 ceph-mon[30108]: osdmap e590: 8 total, 8 up, 8 in 2024-03-24T12:22:49.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:49 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:22:49.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:49 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:22:49.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:49 smithi006 ceph-mon[41734]: osdmap e590: 8 total, 8 up, 8 in 2024-03-24T12:22:49.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:49 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:22:49.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:49 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:22:50.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:50 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-116", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:22:50.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:50 smithi090 ceph-mon[33646]: osdmap e591: 8 total, 8 up, 8 in 2024-03-24T12:22:50.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:50 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-116"}]: dispatch 2024-03-24T12:22:50.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:50 smithi090 ceph-mon[33646]: pgmap v906: 268 pgs: 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:22:50.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:50 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-116", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:22:50.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:50 smithi006 ceph-mon[30108]: osdmap e591: 8 total, 8 up, 8 in 2024-03-24T12:22:50.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:50 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-116"}]: dispatch 2024-03-24T12:22:50.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:50 smithi006 ceph-mon[30108]: pgmap v906: 268 pgs: 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:22:50.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:50 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-116", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:22:50.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:50 smithi006 ceph-mon[41734]: osdmap e591: 8 total, 8 up, 8 in 2024-03-24T12:22:50.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:50 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-116"}]: dispatch 2024-03-24T12:22:50.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:50 smithi006 ceph-mon[41734]: pgmap v906: 268 pgs: 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:22:51.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:51 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-116"}]': finished 2024-03-24T12:22:51.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:51 smithi090 ceph-mon[33646]: osdmap e592: 8 total, 8 up, 8 in 2024-03-24T12:22:51.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:51 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-116", "mode": "writeback"}]: dispatch 2024-03-24T12:22:51.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-116"}]': finished 2024-03-24T12:22:51.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:51 smithi006 ceph-mon[30108]: osdmap e592: 8 total, 8 up, 8 in 2024-03-24T12:22:51.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-116", "mode": "writeback"}]: dispatch 2024-03-24T12:22:51.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:51 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-116"}]': finished 2024-03-24T12:22:51.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:51 smithi006 ceph-mon[41734]: osdmap e592: 8 total, 8 up, 8 in 2024-03-24T12:22:51.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:51 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-116", "mode": "writeback"}]: dispatch 2024-03-24T12:22:52.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:52 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:22:52.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:52 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-116", "mode": "writeback"}]': finished 2024-03-24T12:22:52.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:52 smithi090 ceph-mon[33646]: osdmap e593: 8 total, 8 up, 8 in 2024-03-24T12:22:52.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:52 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "325.3"}]: dispatch 2024-03-24T12:22:52.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:52 smithi090 ceph-mon[33646]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.3"}]: dispatch 2024-03-24T12:22:52.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:52 smithi090 ceph-mon[33646]: pgmap v909: 268 pgs: 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:22:52.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:52 smithi090 ceph-mon[33646]: 325.3 scrub starts 2024-03-24T12:22:52.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:52 smithi090 ceph-mon[33646]: 325.3 scrub ok 2024-03-24T12:22:52.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:52 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:22:52.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:52 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-116", "mode": "writeback"}]': finished 2024-03-24T12:22:52.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:52 smithi006 ceph-mon[30108]: osdmap e593: 8 total, 8 up, 8 in 2024-03-24T12:22:52.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:52 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "325.3"}]: dispatch 2024-03-24T12:22:52.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:52 smithi006 ceph-mon[30108]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.3"}]: dispatch 2024-03-24T12:22:52.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:52 smithi006 ceph-mon[30108]: pgmap v909: 268 pgs: 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:22:52.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:52 smithi006 ceph-mon[30108]: 325.3 scrub starts 2024-03-24T12:22:52.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:52 smithi006 ceph-mon[30108]: 325.3 scrub ok 2024-03-24T12:22:52.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:52 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:22:52.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:52 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-116", "mode": "writeback"}]': finished 2024-03-24T12:22:52.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:52 smithi006 ceph-mon[41734]: osdmap e593: 8 total, 8 up, 8 in 2024-03-24T12:22:52.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:52 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "325.3"}]: dispatch 2024-03-24T12:22:52.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:52 smithi006 ceph-mon[41734]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.3"}]: dispatch 2024-03-24T12:22:52.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:52 smithi006 ceph-mon[41734]: pgmap v909: 268 pgs: 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:22:52.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:52 smithi006 ceph-mon[41734]: 325.3 scrub starts 2024-03-24T12:22:52.463 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:52 smithi006 ceph-mon[41734]: 325.3 scrub ok 2024-03-24T12:22:53.178 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:22:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:22:54.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:54 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:22:54.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:54 smithi090 ceph-mon[33646]: pgmap v910: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 585 B/s wr, 2 op/s 2024-03-24T12:22:54.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:54 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:22:54.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:54 smithi006 ceph-mon[30108]: pgmap v910: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 585 B/s wr, 2 op/s 2024-03-24T12:22:54.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:54 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:22:54.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:54 smithi006 ceph-mon[41734]: pgmap v910: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 585 B/s wr, 2 op/s 2024-03-24T12:22:55.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:22:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:22:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:22:56.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:56 smithi090 ceph-mon[33646]: pgmap v911: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 491 B/s wr, 2 op/s 2024-03-24T12:22:56.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:56 smithi006 ceph-mon[30108]: pgmap v911: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 491 B/s wr, 2 op/s 2024-03-24T12:22:56.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:56 smithi006 ceph-mon[41734]: pgmap v911: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 491 B/s wr, 2 op/s 2024-03-24T12:22:58.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:22:58 smithi090 ceph-mon[33646]: pgmap v912: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 383 B/s wr, 2 op/s 2024-03-24T12:22:58.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:22:58 smithi006 ceph-mon[30108]: pgmap v912: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 383 B/s wr, 2 op/s 2024-03-24T12:22:58.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:22:58 smithi006 ceph-mon[41734]: pgmap v912: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 383 B/s wr, 2 op/s 2024-03-24T12:23:00.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:00 smithi090 ceph-mon[33646]: pgmap v913: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 332 B/s wr, 1 op/s 2024-03-24T12:23:00.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:00 smithi006 ceph-mon[30108]: pgmap v913: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 332 B/s wr, 1 op/s 2024-03-24T12:23:00.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:00 smithi006 ceph-mon[41734]: pgmap v913: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 332 B/s wr, 1 op/s 2024-03-24T12:23:01.568 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.568 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.569 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.569 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.569 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.569 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.569 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.569 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.569 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.569 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.569 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.569 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.569 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.569 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.569 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.569 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.569 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.569 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.570 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.570 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.570 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.570 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.570 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.570 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.570 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.570 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.570 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.570 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.570 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.570 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.570 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.570 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.571 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.571 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.571 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.571 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.571 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.571 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.571 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.571 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.571 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.571 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.571 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.571 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.571 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.571 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.571 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.571 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.572 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.572 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.572 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.572 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.572 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.572 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.572 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.572 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.572 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.572 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.572 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.572 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.572 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.572 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.573 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.573 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.573 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.573 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.573 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.573 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.573 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.573 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.573 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.573 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.573 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.573 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.573 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.573 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.573 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.573 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.574 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.574 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.574 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.574 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.574 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.574 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.574 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.574 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.574 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.574 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.574 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.574 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.574 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.574 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.575 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.575 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.575 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.575 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.575 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.575 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.575 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:23:01.664 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:01 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:23:01.694 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:01 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:23:01.695 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:01 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:23:02.663 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:02 smithi090 ceph-mon[33646]: pgmap v914: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 300 B/s wr, 1 op/s 2024-03-24T12:23:02.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:02 smithi006 ceph-mon[30108]: pgmap v914: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 300 B/s wr, 1 op/s 2024-03-24T12:23:02.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:02 smithi006 ceph-mon[41734]: pgmap v914: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 300 B/s wr, 1 op/s 2024-03-24T12:23:02.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:23:02 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:23:04.838 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:04 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:23:04.838 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:04 smithi090 ceph-mon[33646]: pgmap v915: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:23:04.838 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:04 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:04.838 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:23:04.838 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:04 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:04.838 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:04 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:04.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:04 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:23:04.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:04 smithi006 ceph-mon[41734]: pgmap v915: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:23:04.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:04 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:23:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:04 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:04 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:04 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:23:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:04 smithi006 ceph-mon[30108]: pgmap v915: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:23:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:04 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:23:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:04 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:04.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:04 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:05.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:23:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:23:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:23:06.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:06 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:06.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:06 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:06.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:06 smithi006 ceph-mon[41734]: pgmap v916: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:23:06.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:06 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:23:06.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:06 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:23:06.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:06 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:06.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:06 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:06.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:06 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:06.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:06 smithi006 ceph-mon[30108]: pgmap v916: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:23:06.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:06 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:23:06.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:06 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:23:06.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:06 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:06.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:06 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:06.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:06 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:06.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:06 smithi090 ceph-mon[33646]: pgmap v916: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:23:06.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:06 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:23:06.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:06 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:23:06.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:06 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:08.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:07 smithi090 ceph-mon[33646]: osdmap e594: 8 total, 8 up, 8 in 2024-03-24T12:23:08.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:07 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:23:08.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:07 smithi090 ceph-mon[33646]: pgmap v918: 268 pgs: 268 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:23:08.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:07 smithi006 ceph-mon[41734]: osdmap e594: 8 total, 8 up, 8 in 2024-03-24T12:23:08.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:07 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:23:08.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:07 smithi006 ceph-mon[41734]: pgmap v918: 268 pgs: 268 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:23:08.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:07 smithi006 ceph-mon[30108]: osdmap e594: 8 total, 8 up, 8 in 2024-03-24T12:23:08.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:07 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:23:08.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:07 smithi006 ceph-mon[30108]: pgmap v918: 268 pgs: 268 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:23:09.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:23:09.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:08 smithi090 ceph-mon[33646]: osdmap e595: 8 total, 8 up, 8 in 2024-03-24T12:23:09.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-116"}]: dispatch 2024-03-24T12:23:09.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:08 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:23:09.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-116"}]': finished 2024-03-24T12:23:09.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:23:09.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:08 smithi006 ceph-mon[30108]: osdmap e595: 8 total, 8 up, 8 in 2024-03-24T12:23:09.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-116"}]: dispatch 2024-03-24T12:23:09.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:08 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:23:09.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-116"}]': finished 2024-03-24T12:23:09.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:23:09.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:08 smithi006 ceph-mon[41734]: osdmap e595: 8 total, 8 up, 8 in 2024-03-24T12:23:09.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-116"}]: dispatch 2024-03-24T12:23:09.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:08 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:23:09.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-116"}]': finished 2024-03-24T12:23:10.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:09 smithi090 ceph-mon[33646]: osdmap e596: 8 total, 8 up, 8 in 2024-03-24T12:23:10.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:09 smithi090 ceph-mon[33646]: pgmap v921: 268 pgs: 268 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-24T12:23:10.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:09 smithi006 ceph-mon[30108]: osdmap e596: 8 total, 8 up, 8 in 2024-03-24T12:23:10.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:09 smithi006 ceph-mon[30108]: pgmap v921: 268 pgs: 268 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-24T12:23:10.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:09 smithi006 ceph-mon[41734]: osdmap e596: 8 total, 8 up, 8 in 2024-03-24T12:23:10.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:09 smithi006 ceph-mon[41734]: pgmap v921: 268 pgs: 268 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-24T12:23:11.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:10 smithi090 ceph-mon[33646]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:23:11.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:10 smithi090 ceph-mon[33646]: osdmap e597: 8 total, 8 up, 8 in 2024-03-24T12:23:11.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:10 smithi006 ceph-mon[30108]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:23:11.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:10 smithi006 ceph-mon[30108]: osdmap e597: 8 total, 8 up, 8 in 2024-03-24T12:23:11.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:10 smithi006 ceph-mon[41734]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:23:11.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:10 smithi006 ceph-mon[41734]: osdmap e597: 8 total, 8 up, 8 in 2024-03-24T12:23:12.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:11 smithi006 ceph-mon[30108]: osdmap e598: 8 total, 8 up, 8 in 2024-03-24T12:23:12.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:23:12.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:23:12.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:11 smithi006 ceph-mon[30108]: osdmap e599: 8 total, 8 up, 8 in 2024-03-24T12:23:12.112 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:11 smithi006 ceph-mon[30108]: pgmap v925: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:23:12.112 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:11 smithi006 ceph-mon[41734]: osdmap e598: 8 total, 8 up, 8 in 2024-03-24T12:23:12.112 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:23:12.113 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:23:12.113 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:11 smithi006 ceph-mon[41734]: osdmap e599: 8 total, 8 up, 8 in 2024-03-24T12:23:12.113 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:11 smithi006 ceph-mon[41734]: pgmap v925: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:23:12.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:11 smithi090 ceph-mon[33646]: osdmap e598: 8 total, 8 up, 8 in 2024-03-24T12:23:12.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:23:12.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:23:12.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:11 smithi090 ceph-mon[33646]: osdmap e599: 8 total, 8 up, 8 in 2024-03-24T12:23:12.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:11 smithi090 ceph-mon[33646]: pgmap v925: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:23:13.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:12 smithi090 ceph-mon[33646]: osdmap e600: 8 total, 8 up, 8 in 2024-03-24T12:23:13.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:12 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:23:13.178 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:23:12 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:23:13.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:12 smithi006 ceph-mon[30108]: osdmap e600: 8 total, 8 up, 8 in 2024-03-24T12:23:13.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:12 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:23:13.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:12 smithi006 ceph-mon[41734]: osdmap e600: 8 total, 8 up, 8 in 2024-03-24T12:23:13.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:12 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:23:14.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:13 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:23:14.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:13 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-118", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:23:14.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:13 smithi006 ceph-mon[30108]: osdmap e601: 8 total, 8 up, 8 in 2024-03-24T12:23:14.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:13 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-118"}]: dispatch 2024-03-24T12:23:14.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:13 smithi006 ceph-mon[30108]: pgmap v928: 268 pgs: 268 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 589 B/s wr, 2 op/s 2024-03-24T12:23:14.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:13 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-118"}]': finished 2024-03-24T12:23:14.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:13 smithi006 ceph-mon[30108]: osdmap e602: 8 total, 8 up, 8 in 2024-03-24T12:23:14.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:13 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-118", "mode": "writeback"}]: dispatch 2024-03-24T12:23:14.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:13 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:23:14.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:13 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-118", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:23:14.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:13 smithi006 ceph-mon[41734]: osdmap e601: 8 total, 8 up, 8 in 2024-03-24T12:23:14.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:13 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-118"}]: dispatch 2024-03-24T12:23:14.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:13 smithi006 ceph-mon[41734]: pgmap v928: 268 pgs: 268 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 589 B/s wr, 2 op/s 2024-03-24T12:23:14.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:13 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-118"}]': finished 2024-03-24T12:23:14.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:13 smithi006 ceph-mon[41734]: osdmap e602: 8 total, 8 up, 8 in 2024-03-24T12:23:14.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:13 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-118", "mode": "writeback"}]: dispatch 2024-03-24T12:23:14.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:13 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:23:14.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:13 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-118", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:23:14.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:13 smithi090 ceph-mon[33646]: osdmap e601: 8 total, 8 up, 8 in 2024-03-24T12:23:14.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:13 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-118"}]: dispatch 2024-03-24T12:23:14.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:13 smithi090 ceph-mon[33646]: pgmap v928: 268 pgs: 268 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 589 B/s wr, 2 op/s 2024-03-24T12:23:14.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:13 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-118"}]': finished 2024-03-24T12:23:14.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:13 smithi090 ceph-mon[33646]: osdmap e602: 8 total, 8 up, 8 in 2024-03-24T12:23:14.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:13 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-118", "mode": "writeback"}]: dispatch 2024-03-24T12:23:15.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:14 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:23:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-118", "mode": "writeback"}]': finished 2024-03-24T12:23:15.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:14 smithi006 ceph-mon[30108]: osdmap e603: 8 total, 8 up, 8 in 2024-03-24T12:23:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:14 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:23:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-118", "mode": "writeback"}]': finished 2024-03-24T12:23:15.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:14 smithi006 ceph-mon[41734]: osdmap e603: 8 total, 8 up, 8 in 2024-03-24T12:23:15.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:14 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:23:15.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-118", "mode": "writeback"}]': finished 2024-03-24T12:23:15.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:14 smithi090 ceph-mon[33646]: osdmap e603: 8 total, 8 up, 8 in 2024-03-24T12:23:15.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:23:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:23:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:23:16.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:15 smithi090 ceph-mon[33646]: pgmap v931: 268 pgs: 268 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:23:16.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:15 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:23:16.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:15 smithi090 ceph-mon[33646]: osdmap e604: 8 total, 8 up, 8 in 2024-03-24T12:23:16.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:15 smithi006 ceph-mon[30108]: pgmap v931: 268 pgs: 268 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:23:16.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:15 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:23:16.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:15 smithi006 ceph-mon[30108]: osdmap e604: 8 total, 8 up, 8 in 2024-03-24T12:23:16.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:15 smithi006 ceph-mon[41734]: pgmap v931: 268 pgs: 268 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:23:16.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:15 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:23:16.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:15 smithi006 ceph-mon[41734]: osdmap e604: 8 total, 8 up, 8 in 2024-03-24T12:23:17.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:16 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:23:17.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:16 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:23:17.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:16 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:23:18.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:23:18.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:17 smithi090 ceph-mon[33646]: osdmap e605: 8 total, 8 up, 8 in 2024-03-24T12:23:18.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-118"}]: dispatch 2024-03-24T12:23:18.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:17 smithi090 ceph-mon[33646]: pgmap v934: 268 pgs: 268 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:23:18.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:23:18.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:17 smithi006 ceph-mon[30108]: osdmap e605: 8 total, 8 up, 8 in 2024-03-24T12:23:18.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-118"}]: dispatch 2024-03-24T12:23:18.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:17 smithi006 ceph-mon[30108]: pgmap v934: 268 pgs: 268 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:23:18.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:23:18.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:17 smithi006 ceph-mon[41734]: osdmap e605: 8 total, 8 up, 8 in 2024-03-24T12:23:18.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-118"}]: dispatch 2024-03-24T12:23:18.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:17 smithi006 ceph-mon[41734]: pgmap v934: 268 pgs: 268 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:23:19.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:19 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:23:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-118"}]': finished 2024-03-24T12:23:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:19 smithi090 ceph-mon[33646]: osdmap e606: 8 total, 8 up, 8 in 2024-03-24T12:23:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:19 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:19.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:23:19.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:19 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:23:19.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-118"}]': finished 2024-03-24T12:23:19.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:19 smithi006 ceph-mon[30108]: osdmap e606: 8 total, 8 up, 8 in 2024-03-24T12:23:19.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:19 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:19.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:23:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:19 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:23:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-118"}]': finished 2024-03-24T12:23:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:19 smithi006 ceph-mon[41734]: osdmap e606: 8 total, 8 up, 8 in 2024-03-24T12:23:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:19 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:23:20.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:20 smithi090 ceph-mon[33646]: osdmap e607: 8 total, 8 up, 8 in 2024-03-24T12:23:20.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:20 smithi090 ceph-mon[33646]: pgmap v937: 236 pgs: 236 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:23:20.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:20 smithi090 ceph-mon[33646]: osdmap e608: 8 total, 8 up, 8 in 2024-03-24T12:23:20.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:20 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:23:20.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:20 smithi006 ceph-mon[30108]: osdmap e607: 8 total, 8 up, 8 in 2024-03-24T12:23:20.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:20 smithi006 ceph-mon[30108]: pgmap v937: 236 pgs: 236 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:23:20.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:20 smithi006 ceph-mon[30108]: osdmap e608: 8 total, 8 up, 8 in 2024-03-24T12:23:20.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:20 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:23:20.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:20 smithi006 ceph-mon[41734]: osdmap e607: 8 total, 8 up, 8 in 2024-03-24T12:23:20.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:20 smithi006 ceph-mon[41734]: pgmap v937: 236 pgs: 236 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:23:20.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:20 smithi006 ceph-mon[41734]: osdmap e608: 8 total, 8 up, 8 in 2024-03-24T12:23:20.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:20 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:23:22.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:23:22.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:21 smithi006 ceph-mon[30108]: osdmap e609: 8 total, 8 up, 8 in 2024-03-24T12:23:22.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:23:22.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:21 smithi006 ceph-mon[30108]: pgmap v940: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:23:22.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:23:22.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:21 smithi006 ceph-mon[41734]: osdmap e609: 8 total, 8 up, 8 in 2024-03-24T12:23:22.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:23:22.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:21 smithi006 ceph-mon[41734]: pgmap v940: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:23:22.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:23:22.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:21 smithi090 ceph-mon[33646]: osdmap e609: 8 total, 8 up, 8 in 2024-03-24T12:23:22.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:23:22.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:21 smithi090 ceph-mon[33646]: pgmap v940: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:23:23.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-120", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:23:23.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:22 smithi090 ceph-mon[33646]: osdmap e610: 8 total, 8 up, 8 in 2024-03-24T12:23:23.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-120"}]: dispatch 2024-03-24T12:23:23.178 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:23:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:23:23.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-120", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:23:23.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:22 smithi006 ceph-mon[30108]: osdmap e610: 8 total, 8 up, 8 in 2024-03-24T12:23:23.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-120"}]: dispatch 2024-03-24T12:23:23.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-120", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:23:23.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:22 smithi006 ceph-mon[41734]: osdmap e610: 8 total, 8 up, 8 in 2024-03-24T12:23:23.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-120"}]: dispatch 2024-03-24T12:23:24.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:23 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:23:24.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-120"}]': finished 2024-03-24T12:23:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:23 smithi006 ceph-mon[30108]: osdmap e611: 8 total, 8 up, 8 in 2024-03-24T12:23:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-120", "mode": "writeback"}]: dispatch 2024-03-24T12:23:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:23 smithi006 ceph-mon[30108]: pgmap v943: 268 pgs: 268 active+clean; 582 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:23:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:23 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:23:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-120", "mode": "writeback"}]': finished 2024-03-24T12:23:24.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:23 smithi006 ceph-mon[30108]: osdmap e612: 8 total, 8 up, 8 in 2024-03-24T12:23:24.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:23 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:23:24.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-120"}]': finished 2024-03-24T12:23:24.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:23 smithi006 ceph-mon[41734]: osdmap e611: 8 total, 8 up, 8 in 2024-03-24T12:23:24.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-120", "mode": "writeback"}]: dispatch 2024-03-24T12:23:24.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:23 smithi006 ceph-mon[41734]: pgmap v943: 268 pgs: 268 active+clean; 582 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:23:24.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:23 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:23:24.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-120", "mode": "writeback"}]': finished 2024-03-24T12:23:24.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:23 smithi006 ceph-mon[41734]: osdmap e612: 8 total, 8 up, 8 in 2024-03-24T12:23:24.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:23 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:23:24.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-120"}]': finished 2024-03-24T12:23:24.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:23 smithi090 ceph-mon[33646]: osdmap e611: 8 total, 8 up, 8 in 2024-03-24T12:23:24.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-120", "mode": "writeback"}]: dispatch 2024-03-24T12:23:24.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:23 smithi090 ceph-mon[33646]: pgmap v943: 268 pgs: 268 active+clean; 582 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:23:24.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:23 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:23:24.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-120", "mode": "writeback"}]': finished 2024-03-24T12:23:24.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:23 smithi090 ceph-mon[33646]: osdmap e612: 8 total, 8 up, 8 in 2024-03-24T12:23:25.261 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:23:25.261 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:23:25.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:23:25.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:23:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:23:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:23:26.329 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:23:26.329 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:25 smithi090 ceph-mon[33646]: osdmap e613: 8 total, 8 up, 8 in 2024-03-24T12:23:26.329 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-120"}]: dispatch 2024-03-24T12:23:26.329 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:25 smithi090 ceph-mon[33646]: pgmap v946: 268 pgs: 268 active+clean; 582 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:23:26.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:23:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:25 smithi006 ceph-mon[30108]: osdmap e613: 8 total, 8 up, 8 in 2024-03-24T12:23:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-120"}]: dispatch 2024-03-24T12:23:26.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:25 smithi006 ceph-mon[30108]: pgmap v946: 268 pgs: 268 active+clean; 582 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:23:26.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:23:26.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:25 smithi006 ceph-mon[41734]: osdmap e613: 8 total, 8 up, 8 in 2024-03-24T12:23:26.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-120"}]: dispatch 2024-03-24T12:23:26.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:25 smithi006 ceph-mon[41734]: pgmap v946: 268 pgs: 268 active+clean; 582 KiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:23:27.340 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:26 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:23:27.341 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-120"}]': finished 2024-03-24T12:23:27.341 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:26 smithi090 ceph-mon[33646]: osdmap e614: 8 total, 8 up, 8 in 2024-03-24T12:23:27.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:26 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:23:27.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-120"}]': finished 2024-03-24T12:23:27.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:26 smithi006 ceph-mon[30108]: osdmap e614: 8 total, 8 up, 8 in 2024-03-24T12:23:27.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:26 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:23:27.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-120"}]': finished 2024-03-24T12:23:27.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:26 smithi006 ceph-mon[41734]: osdmap e614: 8 total, 8 up, 8 in 2024-03-24T12:23:28.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:27 smithi090 ceph-mon[33646]: osdmap e615: 8 total, 8 up, 8 in 2024-03-24T12:23:28.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:27 smithi090 ceph-mon[33646]: pgmap v949: 236 pgs: 236 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:23:28.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:27 smithi006 ceph-mon[30108]: osdmap e615: 8 total, 8 up, 8 in 2024-03-24T12:23:28.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:27 smithi006 ceph-mon[30108]: pgmap v949: 236 pgs: 236 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:23:28.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:27 smithi006 ceph-mon[41734]: osdmap e615: 8 total, 8 up, 8 in 2024-03-24T12:23:28.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:27 smithi006 ceph-mon[41734]: pgmap v949: 236 pgs: 236 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:23:29.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:29 smithi090 ceph-mon[33646]: osdmap e616: 8 total, 8 up, 8 in 2024-03-24T12:23:29.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:29 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:23:29.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:29 smithi006 ceph-mon[30108]: osdmap e616: 8 total, 8 up, 8 in 2024-03-24T12:23:29.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:29 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:23:29.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:29 smithi006 ceph-mon[41734]: osdmap e616: 8 total, 8 up, 8 in 2024-03-24T12:23:29.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:29 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:23:30.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:30 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:23:30.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:30 smithi090 ceph-mon[33646]: osdmap e617: 8 total, 8 up, 8 in 2024-03-24T12:23:30.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:30 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:23:30.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:30 smithi090 ceph-mon[33646]: pgmap v952: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:23:30.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:30 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:23:30.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:30 smithi006 ceph-mon[30108]: osdmap e617: 8 total, 8 up, 8 in 2024-03-24T12:23:30.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:30 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:23:30.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:30 smithi006 ceph-mon[30108]: pgmap v952: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:23:30.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:30 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:23:30.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:30 smithi006 ceph-mon[41734]: osdmap e617: 8 total, 8 up, 8 in 2024-03-24T12:23:30.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:30 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:23:30.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:30 smithi006 ceph-mon[41734]: pgmap v952: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:23:31.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-122", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:23:31.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:31 smithi090 ceph-mon[33646]: osdmap e618: 8 total, 8 up, 8 in 2024-03-24T12:23:31.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-122"}]: dispatch 2024-03-24T12:23:31.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-122", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:23:31.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:31 smithi006 ceph-mon[30108]: osdmap e618: 8 total, 8 up, 8 in 2024-03-24T12:23:31.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-122"}]: dispatch 2024-03-24T12:23:31.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-122", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:23:31.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:31 smithi006 ceph-mon[41734]: osdmap e618: 8 total, 8 up, 8 in 2024-03-24T12:23:31.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-122"}]: dispatch 2024-03-24T12:23:32.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:32 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-122"}]': finished 2024-03-24T12:23:32.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:32 smithi090 ceph-mon[33646]: osdmap e619: 8 total, 8 up, 8 in 2024-03-24T12:23:32.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:32 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-122", "mode": "writeback"}]: dispatch 2024-03-24T12:23:32.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:32 smithi090 ceph-mon[33646]: pgmap v955: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:23:32.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:32 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-122"}]': finished 2024-03-24T12:23:32.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:32 smithi006 ceph-mon[30108]: osdmap e619: 8 total, 8 up, 8 in 2024-03-24T12:23:32.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:32 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-122", "mode": "writeback"}]: dispatch 2024-03-24T12:23:32.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:32 smithi006 ceph-mon[30108]: pgmap v955: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:23:32.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:32 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-122"}]': finished 2024-03-24T12:23:32.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:32 smithi006 ceph-mon[41734]: osdmap e619: 8 total, 8 up, 8 in 2024-03-24T12:23:32.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:32 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-122", "mode": "writeback"}]: dispatch 2024-03-24T12:23:32.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:32 smithi006 ceph-mon[41734]: pgmap v955: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:23:33.037 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:23:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:23:33.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:33 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:23:33.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-122", "mode": "writeback"}]': finished 2024-03-24T12:23:33.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:33 smithi090 ceph-mon[33646]: osdmap e620: 8 total, 8 up, 8 in 2024-03-24T12:23:33.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:23:33.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:33 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:23:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-122", "mode": "writeback"}]': finished 2024-03-24T12:23:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:33 smithi006 ceph-mon[30108]: osdmap e620: 8 total, 8 up, 8 in 2024-03-24T12:23:33.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:23:33.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:33 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:23:33.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-122", "mode": "writeback"}]': finished 2024-03-24T12:23:33.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:33 smithi006 ceph-mon[41734]: osdmap e620: 8 total, 8 up, 8 in 2024-03-24T12:23:33.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:23:34.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:34 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:23:34.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:23:34.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:34 smithi090 ceph-mon[33646]: osdmap e621: 8 total, 8 up, 8 in 2024-03-24T12:23:34.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-122"}]: dispatch 2024-03-24T12:23:34.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:34 smithi090 ceph-mon[33646]: pgmap v958: 268 pgs: 268 active+clean; 582 KiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-24T12:23:34.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:34 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:34.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:23:34.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:34 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:23:34.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:34 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-122"}]': finished 2024-03-24T12:23:34.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:34 smithi090 ceph-mon[33646]: osdmap e622: 8 total, 8 up, 8 in 2024-03-24T12:23:34.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:34 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:23:34.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:23:34.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:34 smithi006 ceph-mon[30108]: osdmap e621: 8 total, 8 up, 8 in 2024-03-24T12:23:34.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-122"}]: dispatch 2024-03-24T12:23:34.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:34 smithi006 ceph-mon[30108]: pgmap v958: 268 pgs: 268 active+clean; 582 KiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-24T12:23:34.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:34 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:34.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:23:34.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:34 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:23:34.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:34 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-122"}]': finished 2024-03-24T12:23:34.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:34 smithi006 ceph-mon[30108]: osdmap e622: 8 total, 8 up, 8 in 2024-03-24T12:23:34.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:34 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:23:34.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:23:34.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:34 smithi006 ceph-mon[41734]: osdmap e621: 8 total, 8 up, 8 in 2024-03-24T12:23:34.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-122"}]: dispatch 2024-03-24T12:23:34.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:34 smithi006 ceph-mon[41734]: pgmap v958: 268 pgs: 268 active+clean; 582 KiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-24T12:23:34.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:34 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:34.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:23:34.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:34 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:23:34.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:34 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-122"}]': finished 2024-03-24T12:23:34.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:34 smithi006 ceph-mon[41734]: osdmap e622: 8 total, 8 up, 8 in 2024-03-24T12:23:35.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:23:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:23:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:23:36.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:36 smithi090 ceph-mon[33646]: osdmap e623: 8 total, 8 up, 8 in 2024-03-24T12:23:36.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:36 smithi090 ceph-mon[33646]: pgmap v961: 236 pgs: 236 active+clean; 582 KiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:23:36.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:36 smithi006 ceph-mon[30108]: osdmap e623: 8 total, 8 up, 8 in 2024-03-24T12:23:36.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:36 smithi006 ceph-mon[30108]: pgmap v961: 236 pgs: 236 active+clean; 582 KiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:23:36.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:36 smithi006 ceph-mon[41734]: osdmap e623: 8 total, 8 up, 8 in 2024-03-24T12:23:36.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:36 smithi006 ceph-mon[41734]: pgmap v961: 236 pgs: 236 active+clean; 582 KiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:23:37.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:37 smithi090 ceph-mon[33646]: osdmap e624: 8 total, 8 up, 8 in 2024-03-24T12:23:37.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:23:37.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:37 smithi006 ceph-mon[30108]: osdmap e624: 8 total, 8 up, 8 in 2024-03-24T12:23:37.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:23:37.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:37 smithi006 ceph-mon[41734]: osdmap e624: 8 total, 8 up, 8 in 2024-03-24T12:23:37.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:23:38.430 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:38 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:23:38.431 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:38 smithi090 ceph-mon[33646]: osdmap e625: 8 total, 8 up, 8 in 2024-03-24T12:23:38.431 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:38 smithi090 ceph-mon[33646]: pgmap v964: 268 pgs: 268 active+clean; 582 KiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:23:38.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:38 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:23:38.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:38 smithi006 ceph-mon[30108]: osdmap e625: 8 total, 8 up, 8 in 2024-03-24T12:23:38.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:38 smithi006 ceph-mon[30108]: pgmap v964: 268 pgs: 268 active+clean; 582 KiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:23:38.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:38 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:23:38.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:38 smithi006 ceph-mon[41734]: osdmap e625: 8 total, 8 up, 8 in 2024-03-24T12:23:38.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:38 smithi006 ceph-mon[41734]: pgmap v964: 268 pgs: 268 active+clean; 582 KiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:23:39.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:39 smithi090 ceph-mon[33646]: osdmap e626: 8 total, 8 up, 8 in 2024-03-24T12:23:39.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:39 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:23:39.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:39 smithi006 ceph-mon[30108]: osdmap e626: 8 total, 8 up, 8 in 2024-03-24T12:23:39.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:23:39.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:39 smithi006 ceph-mon[41734]: osdmap e626: 8 total, 8 up, 8 in 2024-03-24T12:23:39.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:39 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:23:40.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-124", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:23:40.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:40 smithi090 ceph-mon[33646]: osdmap e627: 8 total, 8 up, 8 in 2024-03-24T12:23:40.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-124"}]: dispatch 2024-03-24T12:23:40.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:40 smithi090 ceph-mon[33646]: pgmap v967: 268 pgs: 268 active+clean; 582 KiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:23:40.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-124", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:23:40.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:40 smithi006 ceph-mon[30108]: osdmap e627: 8 total, 8 up, 8 in 2024-03-24T12:23:40.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-124"}]: dispatch 2024-03-24T12:23:40.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:40 smithi006 ceph-mon[30108]: pgmap v967: 268 pgs: 268 active+clean; 582 KiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:23:40.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-124", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:23:40.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:40 smithi006 ceph-mon[41734]: osdmap e627: 8 total, 8 up, 8 in 2024-03-24T12:23:40.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-124"}]: dispatch 2024-03-24T12:23:40.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:40 smithi006 ceph-mon[41734]: pgmap v967: 268 pgs: 268 active+clean; 582 KiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:23:41.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-124"}]': finished 2024-03-24T12:23:41.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:41 smithi090 ceph-mon[33646]: osdmap e628: 8 total, 8 up, 8 in 2024-03-24T12:23:41.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-124", "mode": "writeback"}]: dispatch 2024-03-24T12:23:41.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-124"}]': finished 2024-03-24T12:23:41.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:41 smithi006 ceph-mon[30108]: osdmap e628: 8 total, 8 up, 8 in 2024-03-24T12:23:41.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-124", "mode": "writeback"}]: dispatch 2024-03-24T12:23:41.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-124"}]': finished 2024-03-24T12:23:41.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:41 smithi006 ceph-mon[41734]: osdmap e628: 8 total, 8 up, 8 in 2024-03-24T12:23:41.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-124", "mode": "writeback"}]: dispatch 2024-03-24T12:23:42.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:42 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:23:42.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:42 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-124", "mode": "writeback"}]': finished 2024-03-24T12:23:42.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:42 smithi006 ceph-mon[30108]: osdmap e629: 8 total, 8 up, 8 in 2024-03-24T12:23:42.362 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:42 smithi006 ceph-mon[30108]: pgmap v970: 268 pgs: 268 active+clean; 582 KiB data, 673 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:23:42.362 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:42 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:23:42.362 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:42 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-124", "mode": "writeback"}]': finished 2024-03-24T12:23:42.362 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:42 smithi006 ceph-mon[41734]: osdmap e629: 8 total, 8 up, 8 in 2024-03-24T12:23:42.362 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:42 smithi006 ceph-mon[41734]: pgmap v970: 268 pgs: 268 active+clean; 582 KiB data, 673 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:23:42.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:42 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:23:42.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:42 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-124", "mode": "writeback"}]': finished 2024-03-24T12:23:42.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:42 smithi090 ceph-mon[33646]: osdmap e629: 8 total, 8 up, 8 in 2024-03-24T12:23:42.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:42 smithi090 ceph-mon[33646]: pgmap v970: 268 pgs: 268 active+clean; 582 KiB data, 673 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:23:43.122 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:23:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:23:43.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:43 smithi090 ceph-mon[33646]: osdmap e630: 8 total, 8 up, 8 in 2024-03-24T12:23:43.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:43 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:23:43.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:43 smithi006 ceph-mon[30108]: osdmap e630: 8 total, 8 up, 8 in 2024-03-24T12:23:43.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:43 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:23:43.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:43 smithi006 ceph-mon[41734]: osdmap e630: 8 total, 8 up, 8 in 2024-03-24T12:23:43.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:43 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:23:44.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:44 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:23:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:44 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:23:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:44 smithi090 ceph-mon[33646]: osdmap e631: 8 total, 8 up, 8 in 2024-03-24T12:23:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:44 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-124"}]: dispatch 2024-03-24T12:23:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:44 smithi090 ceph-mon[33646]: pgmap v973: 268 pgs: 268 active+clean; 582 KiB data, 674 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-24T12:23:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:44 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:23:44.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:44 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-124"}]': finished 2024-03-24T12:23:44.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:44 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:23:44.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:44 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:23:44.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:44 smithi006 ceph-mon[30108]: osdmap e631: 8 total, 8 up, 8 in 2024-03-24T12:23:44.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:44 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-124"}]: dispatch 2024-03-24T12:23:44.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:44 smithi006 ceph-mon[30108]: pgmap v973: 268 pgs: 268 active+clean; 582 KiB data, 674 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-24T12:23:44.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:44 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:23:44.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:44 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-124"}]': finished 2024-03-24T12:23:44.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:44 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:23:44.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:44 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:23:44.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:44 smithi006 ceph-mon[41734]: osdmap e631: 8 total, 8 up, 8 in 2024-03-24T12:23:44.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:44 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-124"}]: dispatch 2024-03-24T12:23:44.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:44 smithi006 ceph-mon[41734]: pgmap v973: 268 pgs: 268 active+clean; 582 KiB data, 674 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-24T12:23:44.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:44 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:23:44.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:44 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-124"}]': finished 2024-03-24T12:23:45.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:45 smithi090 ceph-mon[33646]: osdmap e632: 8 total, 8 up, 8 in 2024-03-24T12:23:45.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:45 smithi006 ceph-mon[30108]: osdmap e632: 8 total, 8 up, 8 in 2024-03-24T12:23:45.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:45 smithi006 ceph-mon[41734]: osdmap e632: 8 total, 8 up, 8 in 2024-03-24T12:23:45.461 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:23:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:23:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:23:46.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:46 smithi090 ceph-mon[33646]: osdmap e633: 8 total, 8 up, 8 in 2024-03-24T12:23:46.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:46 smithi090 ceph-mon[33646]: pgmap v976: 236 pgs: 236 active+clean; 582 KiB data, 674 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-24T12:23:46.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:46 smithi006 ceph-mon[30108]: osdmap e633: 8 total, 8 up, 8 in 2024-03-24T12:23:46.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:46 smithi006 ceph-mon[30108]: pgmap v976: 236 pgs: 236 active+clean; 582 KiB data, 674 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-24T12:23:46.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:46 smithi006 ceph-mon[41734]: osdmap e633: 8 total, 8 up, 8 in 2024-03-24T12:23:46.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:46 smithi006 ceph-mon[41734]: pgmap v976: 236 pgs: 236 active+clean; 582 KiB data, 674 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-24T12:23:47.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:47 smithi090 ceph-mon[33646]: osdmap e634: 8 total, 8 up, 8 in 2024-03-24T12:23:47.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:23:47.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:47 smithi006 ceph-mon[30108]: osdmap e634: 8 total, 8 up, 8 in 2024-03-24T12:23:47.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:23:47.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:47 smithi006 ceph-mon[41734]: osdmap e634: 8 total, 8 up, 8 in 2024-03-24T12:23:47.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:23:48.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:23:48.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:48 smithi090 ceph-mon[33646]: osdmap e635: 8 total, 8 up, 8 in 2024-03-24T12:23:48.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:23:48.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:48 smithi090 ceph-mon[33646]: pgmap v979: 268 pgs: 8 creating+activating, 20 creating+peering, 240 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:23:48.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-126", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:23:48.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:48 smithi090 ceph-mon[33646]: osdmap e636: 8 total, 8 up, 8 in 2024-03-24T12:23:48.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-126"}]: dispatch 2024-03-24T12:23:48.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:23:48.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:48 smithi006 ceph-mon[30108]: osdmap e635: 8 total, 8 up, 8 in 2024-03-24T12:23:48.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:23:48.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:48 smithi006 ceph-mon[30108]: pgmap v979: 268 pgs: 8 creating+activating, 20 creating+peering, 240 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:23:48.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-126", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:23:48.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:48 smithi006 ceph-mon[30108]: osdmap e636: 8 total, 8 up, 8 in 2024-03-24T12:23:48.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-126"}]: dispatch 2024-03-24T12:23:48.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:23:48.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:48 smithi006 ceph-mon[41734]: osdmap e635: 8 total, 8 up, 8 in 2024-03-24T12:23:48.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:23:48.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:48 smithi006 ceph-mon[41734]: pgmap v979: 268 pgs: 8 creating+activating, 20 creating+peering, 240 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:23:48.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-126", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:23:48.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:48 smithi006 ceph-mon[41734]: osdmap e636: 8 total, 8 up, 8 in 2024-03-24T12:23:48.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-126"}]: dispatch 2024-03-24T12:23:49.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:49 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:49.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:49 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:23:49.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:49 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-126"}]': finished 2024-03-24T12:23:49.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:49 smithi090 ceph-mon[33646]: osdmap e637: 8 total, 8 up, 8 in 2024-03-24T12:23:49.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:49 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-126", "mode": "writeback"}]: dispatch 2024-03-24T12:23:49.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:49 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:49.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:49 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:23:49.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:49 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-126"}]': finished 2024-03-24T12:23:49.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:49 smithi006 ceph-mon[30108]: osdmap e637: 8 total, 8 up, 8 in 2024-03-24T12:23:49.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:49 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-126", "mode": "writeback"}]: dispatch 2024-03-24T12:23:49.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:49 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:23:49.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:49 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:23:49.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:49 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-126"}]': finished 2024-03-24T12:23:49.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:49 smithi006 ceph-mon[41734]: osdmap e637: 8 total, 8 up, 8 in 2024-03-24T12:23:49.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:49 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-126", "mode": "writeback"}]: dispatch 2024-03-24T12:23:50.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:50 smithi090 ceph-mon[33646]: pgmap v982: 268 pgs: 8 creating+activating, 20 creating+peering, 240 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:23:50.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:50 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:23:50.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:50 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-126", "mode": "writeback"}]': finished 2024-03-24T12:23:50.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:50 smithi090 ceph-mon[33646]: osdmap e638: 8 total, 8 up, 8 in 2024-03-24T12:23:50.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:50 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:23:50.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:50 smithi006 ceph-mon[30108]: pgmap v982: 268 pgs: 8 creating+activating, 20 creating+peering, 240 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:23:50.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:50 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:23:50.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:50 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-126", "mode": "writeback"}]': finished 2024-03-24T12:23:50.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:50 smithi006 ceph-mon[30108]: osdmap e638: 8 total, 8 up, 8 in 2024-03-24T12:23:50.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:50 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:23:50.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:50 smithi006 ceph-mon[41734]: pgmap v982: 268 pgs: 8 creating+activating, 20 creating+peering, 240 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:23:50.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:50 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:23:50.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:50 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-126", "mode": "writeback"}]': finished 2024-03-24T12:23:50.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:50 smithi006 ceph-mon[41734]: osdmap e638: 8 total, 8 up, 8 in 2024-03-24T12:23:50.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:50 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:23:52.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:23:52.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:51 smithi006 ceph-mon[30108]: osdmap e639: 8 total, 8 up, 8 in 2024-03-24T12:23:52.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-126"}]: dispatch 2024-03-24T12:23:52.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:51 smithi006 ceph-mon[30108]: pgmap v985: 268 pgs: 8 creating+activating, 20 creating+peering, 240 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:23:52.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:51 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:23:52.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:51 smithi006 ceph-mon[41734]: osdmap e639: 8 total, 8 up, 8 in 2024-03-24T12:23:52.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:51 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-126"}]: dispatch 2024-03-24T12:23:52.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:51 smithi006 ceph-mon[41734]: pgmap v985: 268 pgs: 8 creating+activating, 20 creating+peering, 240 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:23:52.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:51 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:23:52.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:51 smithi090 ceph-mon[33646]: osdmap e639: 8 total, 8 up, 8 in 2024-03-24T12:23:52.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:51 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-126"}]: dispatch 2024-03-24T12:23:52.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:51 smithi090 ceph-mon[33646]: pgmap v985: 268 pgs: 8 creating+activating, 20 creating+peering, 240 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:23:52.965 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 seapi_tier_pp: require_osd_release = squid 2024-03-24T12:23:52.966 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9065 ms) 2024-03-24T12:23:52.966 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-03-24T12:23:52.966 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: require_osd_release = squid 2024-03-24T12:23:52.966 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6051 ms) 2024-03-24T12:23:52.966 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-03-24T12:23:52.966 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: require_osd_release = squid 2024-03-24T12:23:52.966 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5048 ms) 2024-03-24T12:23:52.966 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-03-24T12:23:52.966 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: require_osd_release = squid 2024-03-24T12:23:52.966 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (25552 ms) 2024-03-24T12:23:52.966 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-03-24T12:23:52.967 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: require_osd_release = squid 2024-03-24T12:23:52.967 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (14061 ms) 2024-03-24T12:23:52.967 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-03-24T12:23:52.967 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12089 ms) 2024-03-24T12:23:52.967 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-03-24T12:23:52.967 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-03-24T12:23:52.967 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: 00000010 69 73 |is| 2024-03-24T12:23:52.967 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: 00000012 2024-03-24T12:23:52.967 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12083 ms) 2024-03-24T12:23:52.967 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-03-24T12:23:52.967 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: require_osd_release = squid 2024-03-24T12:23:52.967 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6040 ms) 2024-03-24T12:23:52.967 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (553811 ms total) 2024-03-24T12:23:52.967 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: 2024-03-24T12:23:52.967 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-03-24T12:23:52.968 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-03-24T12:23:52.968 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1015 ms) 2024-03-24T12:23:52.968 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-03-24T12:23:52.968 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (11068 ms) 2024-03-24T12:23:52.968 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-03-24T12:23:52.968 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18133 ms) 2024-03-24T12:23:52.968 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-03-24T12:23:52.968 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (3 ms) 2024-03-24T12:23:52.968 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30219 ms total) 2024-03-24T12:23:52.968 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: 2024-03-24T12:23:52.968 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-03-24T12:23:52.968 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-03-24T12:23:52.968 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7049 ms) 2024-03-24T12:23:52.968 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-03-24T12:23:52.968 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8049 ms) 2024-03-24T12:23:52.969 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-03-24T12:23:52.969 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: waiting for scrub... 2024-03-24T12:23:52.969 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: done waiting 2024-03-24T12:23:52.969 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24797 ms) 2024-03-24T12:23:52.969 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-03-24T12:23:52.969 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (9191 ms) 2024-03-24T12:23:52.969 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-03-24T12:23:52.969 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (8002 ms) 2024-03-24T12:23:52.969 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-03-24T12:23:52.969 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8063 ms) 2024-03-24T12:23:52.969 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-03-24T12:23:52.969 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10085 ms) 2024-03-24T12:23:52.969 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-03-24T12:23:53.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:52 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:23:53.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:52 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-126"}]': finished 2024-03-24T12:23:53.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:52 smithi090 ceph-mon[33646]: osdmap e640: 8 total, 8 up, 8 in 2024-03-24T12:23:53.178 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:23:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:23:53.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:52 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:23:53.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:52 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-126"}]': finished 2024-03-24T12:23:53.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:52 smithi006 ceph-mon[30108]: osdmap e640: 8 total, 8 up, 8 in 2024-03-24T12:23:53.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:52 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:23:53.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:52 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-126"}]': finished 2024-03-24T12:23:53.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:52 smithi006 ceph-mon[41734]: osdmap e640: 8 total, 8 up, 8 in 2024-03-24T12:23:54.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:53 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:23:54.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:53 smithi090 ceph-mon[33646]: osdmap e641: 8 total, 8 up, 8 in 2024-03-24T12:23:54.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:53 smithi090 ceph-mon[33646]: pgmap v988: 236 pgs: 236 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:23:54.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:53 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:23:54.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:53 smithi006 ceph-mon[30108]: osdmap e641: 8 total, 8 up, 8 in 2024-03-24T12:23:54.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:53 smithi006 ceph-mon[30108]: pgmap v988: 236 pgs: 236 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:23:54.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:53 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:23:54.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:53 smithi006 ceph-mon[41734]: osdmap e641: 8 total, 8 up, 8 in 2024-03-24T12:23:54.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:53 smithi006 ceph-mon[41734]: pgmap v988: 236 pgs: 236 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:23:55.260 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:54 smithi006 ceph-mon[30108]: osdmap e642: 8 total, 8 up, 8 in 2024-03-24T12:23:55.260 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:23:55.260 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:54 smithi006 ceph-mon[41734]: osdmap e642: 8 total, 8 up, 8 in 2024-03-24T12:23:55.260 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:23:55.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:54 smithi090 ceph-mon[33646]: osdmap e642: 8 total, 8 up, 8 in 2024-03-24T12:23:55.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:23:55.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:23:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:23:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:23:56.329 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:55 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:23:56.329 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:55 smithi090 ceph-mon[33646]: osdmap e643: 8 total, 8 up, 8 in 2024-03-24T12:23:56.329 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:55 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:23:56.329 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:55 smithi090 ceph-mon[33646]: pgmap v991: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:23:56.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:55 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:23:56.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:55 smithi006 ceph-mon[30108]: osdmap e643: 8 total, 8 up, 8 in 2024-03-24T12:23:56.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:55 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:23:56.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:55 smithi006 ceph-mon[30108]: pgmap v991: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:23:56.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:55 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:23:56.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:55 smithi006 ceph-mon[41734]: osdmap e643: 8 total, 8 up, 8 in 2024-03-24T12:23:56.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:55 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:23:56.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:55 smithi006 ceph-mon[41734]: pgmap v991: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:23:57.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:56 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-128", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:23:57.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:56 smithi090 ceph-mon[33646]: osdmap e644: 8 total, 8 up, 8 in 2024-03-24T12:23:57.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:56 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-128"}]: dispatch 2024-03-24T12:23:57.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:56 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-128", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:23:57.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:56 smithi006 ceph-mon[30108]: osdmap e644: 8 total, 8 up, 8 in 2024-03-24T12:23:57.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:56 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-128"}]: dispatch 2024-03-24T12:23:57.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:56 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-128", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:23:57.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:56 smithi006 ceph-mon[41734]: osdmap e644: 8 total, 8 up, 8 in 2024-03-24T12:23:57.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:56 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-128"}]: dispatch 2024-03-24T12:23:58.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:58 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-128"}]': finished 2024-03-24T12:23:58.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:58 smithi090 ceph-mon[33646]: osdmap e645: 8 total, 8 up, 8 in 2024-03-24T12:23:58.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:58 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-128", "mode": "writeback"}]: dispatch 2024-03-24T12:23:58.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:58 smithi090 ceph-mon[33646]: pgmap v994: 268 pgs: 268 active+clean; 582 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:23:58.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:58 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-128"}]': finished 2024-03-24T12:23:58.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:58 smithi006 ceph-mon[30108]: osdmap e645: 8 total, 8 up, 8 in 2024-03-24T12:23:58.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:58 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-128", "mode": "writeback"}]: dispatch 2024-03-24T12:23:58.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:58 smithi006 ceph-mon[30108]: pgmap v994: 268 pgs: 268 active+clean; 582 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:23:58.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:58 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-128"}]': finished 2024-03-24T12:23:58.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:58 smithi006 ceph-mon[41734]: osdmap e645: 8 total, 8 up, 8 in 2024-03-24T12:23:58.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:58 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-128", "mode": "writeback"}]: dispatch 2024-03-24T12:23:58.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:58 smithi006 ceph-mon[41734]: pgmap v994: 268 pgs: 268 active+clean; 582 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:23:59.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:59 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:23:59.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:59 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-128", "mode": "writeback"}]': finished 2024-03-24T12:23:59.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:23:59 smithi090 ceph-mon[33646]: osdmap e646: 8 total, 8 up, 8 in 2024-03-24T12:23:59.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:59 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:23:59.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:59 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-128", "mode": "writeback"}]': finished 2024-03-24T12:23:59.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:23:59 smithi006 ceph-mon[30108]: osdmap e646: 8 total, 8 up, 8 in 2024-03-24T12:23:59.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:59 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:23:59.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:59 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-128", "mode": "writeback"}]': finished 2024-03-24T12:23:59.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:23:59 smithi006 ceph-mon[41734]: osdmap e646: 8 total, 8 up, 8 in 2024-03-24T12:24:00.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:00 smithi090 ceph-mon[33646]: pgmap v996: 268 pgs: 268 active+clean; 582 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:00.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:00 smithi006 ceph-mon[30108]: pgmap v996: 268 pgs: 268 active+clean; 582 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:00.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:00 smithi006 ceph-mon[41734]: pgmap v996: 268 pgs: 268 active+clean; 582 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:02.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:02 smithi090 ceph-mon[33646]: pgmap v997: 268 pgs: 268 active+clean; 582 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:24:02.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:02 smithi006 ceph-mon[30108]: pgmap v997: 268 pgs: 268 active+clean; 582 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:24:02.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:02 smithi006 ceph-mon[41734]: pgmap v997: 268 pgs: 268 active+clean; 582 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:24:03.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:24:02 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:24:03.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:24:03.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:24:03.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:24:04.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:04 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:24:04.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:04 smithi090 ceph-mon[33646]: pgmap v998: 268 pgs: 268 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 281 B/s wr, 1 op/s 2024-03-24T12:24:04.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:04 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:24:04.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:04 smithi090 ceph-mon[33646]: osdmap e647: 8 total, 8 up, 8 in 2024-03-24T12:24:04.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:04 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-128"}]: dispatch 2024-03-24T12:24:04.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:04 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:24:04.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:24:04.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:04 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:24:04.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:04 smithi006 ceph-mon[30108]: pgmap v998: 268 pgs: 268 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 281 B/s wr, 1 op/s 2024-03-24T12:24:04.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:04 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:24:04.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:04 smithi006 ceph-mon[30108]: osdmap e647: 8 total, 8 up, 8 in 2024-03-24T12:24:04.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:04 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-128"}]: dispatch 2024-03-24T12:24:04.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:04 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:24:04.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:24:04.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:04 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:24:04.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:04 smithi006 ceph-mon[41734]: pgmap v998: 268 pgs: 268 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 281 B/s wr, 1 op/s 2024-03-24T12:24:04.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:04 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:24:04.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:04 smithi006 ceph-mon[41734]: osdmap e647: 8 total, 8 up, 8 in 2024-03-24T12:24:04.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:04 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-128"}]: dispatch 2024-03-24T12:24:04.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:04 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:24:04.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:24:05.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:05 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:24:05.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:05 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-128"}]': finished 2024-03-24T12:24:05.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:05 smithi090 ceph-mon[33646]: osdmap e648: 8 total, 8 up, 8 in 2024-03-24T12:24:05.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:24:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:24:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:24:05.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:05 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:24:05.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:05 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-128"}]': finished 2024-03-24T12:24:05.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:05 smithi006 ceph-mon[30108]: osdmap e648: 8 total, 8 up, 8 in 2024-03-24T12:24:05.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:05 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:24:05.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:05 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-128"}]': finished 2024-03-24T12:24:05.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:05 smithi006 ceph-mon[41734]: osdmap e648: 8 total, 8 up, 8 in 2024-03-24T12:24:06.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:06 smithi090 ceph-mon[33646]: pgmap v1001: 268 pgs: 268 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 845 B/s rd, 281 B/s wr, 1 op/s 2024-03-24T12:24:06.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:06 smithi090 ceph-mon[33646]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:06.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:06 smithi090 ceph-mon[33646]: osdmap e649: 8 total, 8 up, 8 in 2024-03-24T12:24:06.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:06 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:24:06.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:06 smithi006 ceph-mon[30108]: pgmap v1001: 268 pgs: 268 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 845 B/s rd, 281 B/s wr, 1 op/s 2024-03-24T12:24:06.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:06 smithi006 ceph-mon[30108]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:06.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:06 smithi006 ceph-mon[30108]: osdmap e649: 8 total, 8 up, 8 in 2024-03-24T12:24:06.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:06 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:24:06.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:06 smithi006 ceph-mon[41734]: pgmap v1001: 268 pgs: 268 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 845 B/s rd, 281 B/s wr, 1 op/s 2024-03-24T12:24:06.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:06 smithi006 ceph-mon[41734]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:06.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:06 smithi006 ceph-mon[41734]: osdmap e649: 8 total, 8 up, 8 in 2024-03-24T12:24:06.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:06 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:24:07.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:07 smithi090 ceph-mon[33646]: osdmap e650: 8 total, 8 up, 8 in 2024-03-24T12:24:07.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:07 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:24:07.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:07 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:24:07.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:07 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:24:07.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:07 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:24:07.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:07 smithi006 ceph-mon[30108]: osdmap e650: 8 total, 8 up, 8 in 2024-03-24T12:24:07.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:07 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:24:07.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:07 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:24:07.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:07 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:24:07.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:07 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:24:07.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:07 smithi006 ceph-mon[41734]: osdmap e650: 8 total, 8 up, 8 in 2024-03-24T12:24:07.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:07 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:24:07.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:07 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:24:07.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:07 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:24:07.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:07 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:24:08.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:08 smithi090 ceph-mon[33646]: pgmap v1004: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:24:08.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:24:08.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:08 smithi090 ceph-mon[33646]: osdmap e651: 8 total, 8 up, 8 in 2024-03-24T12:24:08.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:24:08.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:08 smithi006 ceph-mon[30108]: pgmap v1004: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:24:08.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:24:08.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:08 smithi006 ceph-mon[30108]: osdmap e651: 8 total, 8 up, 8 in 2024-03-24T12:24:08.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:24:08.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:08 smithi006 ceph-mon[41734]: pgmap v1004: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:24:08.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:24:08.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:08 smithi006 ceph-mon[41734]: osdmap e651: 8 total, 8 up, 8 in 2024-03-24T12:24:08.712 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:24:09.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:09 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-130", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:24:09.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:09 smithi090 ceph-mon[33646]: osdmap e652: 8 total, 8 up, 8 in 2024-03-24T12:24:09.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:09 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-130"}]: dispatch 2024-03-24T12:24:09.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:09 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-130"}]': finished 2024-03-24T12:24:09.679 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:09 smithi090 ceph-mon[33646]: osdmap e653: 8 total, 8 up, 8 in 2024-03-24T12:24:09.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:09 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-130", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:24:09.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:09 smithi006 ceph-mon[30108]: osdmap e652: 8 total, 8 up, 8 in 2024-03-24T12:24:09.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:09 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-130"}]: dispatch 2024-03-24T12:24:09.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:09 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-130"}]': finished 2024-03-24T12:24:09.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:09 smithi006 ceph-mon[30108]: osdmap e653: 8 total, 8 up, 8 in 2024-03-24T12:24:09.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:09 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-130", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:24:09.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:09 smithi006 ceph-mon[41734]: osdmap e652: 8 total, 8 up, 8 in 2024-03-24T12:24:09.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:09 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-130"}]: dispatch 2024-03-24T12:24:09.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:09 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-130"}]': finished 2024-03-24T12:24:09.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:09 smithi006 ceph-mon[41734]: osdmap e653: 8 total, 8 up, 8 in 2024-03-24T12:24:10.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:10 smithi090 ceph-mon[33646]: pgmap v1007: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:24:10.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-130", "mode": "writeback"}]: dispatch 2024-03-24T12:24:10.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:10 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:24:10.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-130", "mode": "writeback"}]': finished 2024-03-24T12:24:10.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:10 smithi090 ceph-mon[33646]: osdmap e654: 8 total, 8 up, 8 in 2024-03-24T12:24:10.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:10 smithi006 ceph-mon[30108]: pgmap v1007: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:24:10.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-130", "mode": "writeback"}]: dispatch 2024-03-24T12:24:10.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:10 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:24:10.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-130", "mode": "writeback"}]': finished 2024-03-24T12:24:10.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:10 smithi006 ceph-mon[30108]: osdmap e654: 8 total, 8 up, 8 in 2024-03-24T12:24:10.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:10 smithi006 ceph-mon[41734]: pgmap v1007: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:24:10.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-130", "mode": "writeback"}]: dispatch 2024-03-24T12:24:10.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:10 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:24:10.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-130", "mode": "writeback"}]': finished 2024-03-24T12:24:10.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:10 smithi006 ceph-mon[41734]: osdmap e654: 8 total, 8 up, 8 in 2024-03-24T12:24:11.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:24:11.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:11 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:11.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:24:11.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:11 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:11.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:24:11.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:11 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:12.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:12 smithi090 ceph-mon[33646]: pgmap v1010: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:24:12.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:12 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:24:12.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:12 smithi090 ceph-mon[33646]: osdmap e655: 8 total, 8 up, 8 in 2024-03-24T12:24:12.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:12 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-130"}]: dispatch 2024-03-24T12:24:12.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:12 smithi006 ceph-mon[30108]: pgmap v1010: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:24:12.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:12 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:24:12.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:12 smithi006 ceph-mon[30108]: osdmap e655: 8 total, 8 up, 8 in 2024-03-24T12:24:12.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:12 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-130"}]: dispatch 2024-03-24T12:24:12.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:12 smithi006 ceph-mon[41734]: pgmap v1010: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 702 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:24:12.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:12 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:24:12.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:12 smithi006 ceph-mon[41734]: osdmap e655: 8 total, 8 up, 8 in 2024-03-24T12:24:12.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:12 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-130"}]: dispatch 2024-03-24T12:24:13.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:24:12 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:24:13.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:13 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:24:13.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:13 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-130"}]': finished 2024-03-24T12:24:13.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:13 smithi090 ceph-mon[33646]: osdmap e656: 8 total, 8 up, 8 in 2024-03-24T12:24:13.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:13 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:24:13.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:13 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-130"}]': finished 2024-03-24T12:24:13.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:13 smithi006 ceph-mon[30108]: osdmap e656: 8 total, 8 up, 8 in 2024-03-24T12:24:13.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:13 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:24:13.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:13 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-130"}]': finished 2024-03-24T12:24:13.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:13 smithi006 ceph-mon[41734]: osdmap e656: 8 total, 8 up, 8 in 2024-03-24T12:24:14.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:14 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:24:14.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:14 smithi090 ceph-mon[33646]: pgmap v1013: 268 pgs: 268 active+clean; 582 KiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-03-24T12:24:14.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:14 smithi090 ceph-mon[33646]: osdmap e657: 8 total, 8 up, 8 in 2024-03-24T12:24:14.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:14 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:24:14.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:14 smithi006 ceph-mon[30108]: pgmap v1013: 268 pgs: 268 active+clean; 582 KiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-03-24T12:24:14.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:14 smithi006 ceph-mon[30108]: osdmap e657: 8 total, 8 up, 8 in 2024-03-24T12:24:14.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:14 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:24:14.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:14 smithi006 ceph-mon[41734]: pgmap v1013: 268 pgs: 268 active+clean; 582 KiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-03-24T12:24:14.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:14 smithi006 ceph-mon[41734]: osdmap e657: 8 total, 8 up, 8 in 2024-03-24T12:24:15.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:15 smithi090 ceph-mon[33646]: osdmap e658: 8 total, 8 up, 8 in 2024-03-24T12:24:15.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:24:15.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:24:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:24:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:24:15.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:15 smithi006 ceph-mon[30108]: osdmap e658: 8 total, 8 up, 8 in 2024-03-24T12:24:15.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:24:15.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:15 smithi006 ceph-mon[41734]: osdmap e658: 8 total, 8 up, 8 in 2024-03-24T12:24:15.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:24:16.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:16 smithi090 ceph-mon[33646]: pgmap v1016: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:24:16.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:16 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:24:16.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:16 smithi090 ceph-mon[33646]: osdmap e659: 8 total, 8 up, 8 in 2024-03-24T12:24:16.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:16 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:24:16.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:16 smithi090 ceph-mon[33646]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:16.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:16 smithi006 ceph-mon[30108]: pgmap v1016: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:24:16.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:16 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:24:16.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:16 smithi006 ceph-mon[30108]: osdmap e659: 8 total, 8 up, 8 in 2024-03-24T12:24:16.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:16 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:24:16.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:16 smithi006 ceph-mon[30108]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:16.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:16 smithi006 ceph-mon[41734]: pgmap v1016: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 707 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-24T12:24:16.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:16 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:24:16.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:16 smithi006 ceph-mon[41734]: osdmap e659: 8 total, 8 up, 8 in 2024-03-24T12:24:16.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:16 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:24:16.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:16 smithi006 ceph-mon[41734]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:17.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-132", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:24:17.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:17 smithi006 ceph-mon[30108]: osdmap e660: 8 total, 8 up, 8 in 2024-03-24T12:24:17.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-132"}]: dispatch 2024-03-24T12:24:17.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-132"}]': finished 2024-03-24T12:24:17.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:17 smithi006 ceph-mon[30108]: osdmap e661: 8 total, 8 up, 8 in 2024-03-24T12:24:17.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-132", "mode": "writeback"}]: dispatch 2024-03-24T12:24:17.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-132", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:24:17.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:17 smithi006 ceph-mon[41734]: osdmap e660: 8 total, 8 up, 8 in 2024-03-24T12:24:17.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-132"}]: dispatch 2024-03-24T12:24:17.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-132"}]': finished 2024-03-24T12:24:17.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:17 smithi006 ceph-mon[41734]: osdmap e661: 8 total, 8 up, 8 in 2024-03-24T12:24:17.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-132", "mode": "writeback"}]: dispatch 2024-03-24T12:24:17.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-132", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:24:17.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:17 smithi090 ceph-mon[33646]: osdmap e660: 8 total, 8 up, 8 in 2024-03-24T12:24:17.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-132"}]: dispatch 2024-03-24T12:24:17.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-132"}]': finished 2024-03-24T12:24:17.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:17 smithi090 ceph-mon[33646]: osdmap e661: 8 total, 8 up, 8 in 2024-03-24T12:24:17.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-132", "mode": "writeback"}]: dispatch 2024-03-24T12:24:18.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:18 smithi006 ceph-mon[30108]: pgmap v1019: 268 pgs: 268 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:24:18.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:18 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:24:18.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-132", "mode": "writeback"}]': finished 2024-03-24T12:24:18.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:18 smithi006 ceph-mon[30108]: osdmap e662: 8 total, 8 up, 8 in 2024-03-24T12:24:18.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:18 smithi006 ceph-mon[41734]: pgmap v1019: 268 pgs: 268 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:24:18.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:18 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:24:18.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-132", "mode": "writeback"}]': finished 2024-03-24T12:24:18.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:18 smithi006 ceph-mon[41734]: osdmap e662: 8 total, 8 up, 8 in 2024-03-24T12:24:18.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:18 smithi090 ceph-mon[33646]: pgmap v1019: 268 pgs: 268 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:24:18.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:18 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:24:18.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-132", "mode": "writeback"}]': finished 2024-03-24T12:24:18.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:18 smithi090 ceph-mon[33646]: osdmap e662: 8 total, 8 up, 8 in 2024-03-24T12:24:19.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:24:19.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:19 smithi006 ceph-mon[30108]: osdmap e663: 8 total, 8 up, 8 in 2024-03-24T12:24:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:24:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:19 smithi006 ceph-mon[41734]: osdmap e663: 8 total, 8 up, 8 in 2024-03-24T12:24:19.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:24:19.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:19 smithi090 ceph-mon[33646]: osdmap e663: 8 total, 8 up, 8 in 2024-03-24T12:24:20.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:20 smithi006 ceph-mon[30108]: pgmap v1022: 268 pgs: 268 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:20.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:20 smithi006 ceph-mon[30108]: osdmap e664: 8 total, 8 up, 8 in 2024-03-24T12:24:20.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:20 smithi006 ceph-mon[41734]: pgmap v1022: 268 pgs: 268 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:20.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:20 smithi006 ceph-mon[41734]: osdmap e664: 8 total, 8 up, 8 in 2024-03-24T12:24:20.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:20 smithi090 ceph-mon[33646]: pgmap v1022: 268 pgs: 268 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:20.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:20 smithi090 ceph-mon[33646]: osdmap e664: 8 total, 8 up, 8 in 2024-03-24T12:24:21.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:24:21.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:21 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:21.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:24:21.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:21 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:21.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:24:21.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:21 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:22.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:22 smithi006 ceph-mon[30108]: pgmap v1025: 268 pgs: 268 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:24:22.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:24:22.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:22 smithi006 ceph-mon[30108]: osdmap e665: 8 total, 8 up, 8 in 2024-03-24T12:24:22.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-132"}]: dispatch 2024-03-24T12:24:22.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:22 smithi006 ceph-mon[41734]: pgmap v1025: 268 pgs: 268 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:24:22.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:24:22.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:22 smithi006 ceph-mon[41734]: osdmap e665: 8 total, 8 up, 8 in 2024-03-24T12:24:22.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-132"}]: dispatch 2024-03-24T12:24:22.806 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:22 smithi090 ceph-mon[33646]: pgmap v1025: 268 pgs: 268 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:24:22.806 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:24:22.806 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:22 smithi090 ceph-mon[33646]: osdmap e665: 8 total, 8 up, 8 in 2024-03-24T12:24:22.806 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-132"}]: dispatch 2024-03-24T12:24:23.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:24:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:24:23.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-132"}]': finished 2024-03-24T12:24:23.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:23 smithi006 ceph-mon[30108]: osdmap e666: 8 total, 8 up, 8 in 2024-03-24T12:24:23.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-132"}]': finished 2024-03-24T12:24:23.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:23 smithi006 ceph-mon[41734]: osdmap e666: 8 total, 8 up, 8 in 2024-03-24T12:24:23.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-132"}]': finished 2024-03-24T12:24:23.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:23 smithi090 ceph-mon[33646]: osdmap e666: 8 total, 8 up, 8 in 2024-03-24T12:24:24.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:24 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:24:24.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:24 smithi090 ceph-mon[33646]: pgmap v1028: 268 pgs: 268 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-24T12:24:24.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:24 smithi090 ceph-mon[33646]: osdmap e667: 8 total, 8 up, 8 in 2024-03-24T12:24:24.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:24:24.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:24 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:24:24.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:24 smithi006 ceph-mon[30108]: pgmap v1028: 268 pgs: 268 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-24T12:24:24.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:24 smithi006 ceph-mon[30108]: osdmap e667: 8 total, 8 up, 8 in 2024-03-24T12:24:24.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:24:24.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:24 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:24:24.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:24 smithi006 ceph-mon[41734]: pgmap v1028: 268 pgs: 268 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-24T12:24:24.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:24 smithi006 ceph-mon[41734]: osdmap e667: 8 total, 8 up, 8 in 2024-03-24T12:24:24.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:24:25.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:24:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:24:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:24:25.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:24:25.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:25 smithi006 ceph-mon[30108]: osdmap e668: 8 total, 8 up, 8 in 2024-03-24T12:24:25.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-132"}]: dispatch 2024-03-24T12:24:25.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:25 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:24:25.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-132"}]': finished 2024-03-24T12:24:25.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:24:25.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:25 smithi006 ceph-mon[41734]: osdmap e668: 8 total, 8 up, 8 in 2024-03-24T12:24:25.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-132"}]: dispatch 2024-03-24T12:24:25.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:25 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:24:25.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-132"}]': finished 2024-03-24T12:24:25.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:24:25.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:25 smithi090 ceph-mon[33646]: osdmap e668: 8 total, 8 up, 8 in 2024-03-24T12:24:25.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-132"}]: dispatch 2024-03-24T12:24:25.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:25 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:24:25.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-132"}]': finished 2024-03-24T12:24:27.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:26 smithi090 ceph-mon[33646]: pgmap v1031: 268 pgs: 268 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-24T12:24:27.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:26 smithi090 ceph-mon[33646]: osdmap e669: 8 total, 8 up, 8 in 2024-03-24T12:24:27.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:26 smithi006 ceph-mon[30108]: pgmap v1031: 268 pgs: 268 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-24T12:24:27.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:26 smithi006 ceph-mon[30108]: osdmap e669: 8 total, 8 up, 8 in 2024-03-24T12:24:27.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:26 smithi006 ceph-mon[41734]: pgmap v1031: 268 pgs: 268 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-24T12:24:27.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:26 smithi006 ceph-mon[41734]: osdmap e669: 8 total, 8 up, 8 in 2024-03-24T12:24:28.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:27 smithi090 ceph-mon[33646]: osdmap e670: 8 total, 8 up, 8 in 2024-03-24T12:24:28.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:27 smithi090 ceph-mon[33646]: pgmap v1034: 236 pgs: 236 active+clean; 582 KiB data, 713 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-24T12:24:28.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:27 smithi090 ceph-mon[33646]: osdmap e671: 8 total, 8 up, 8 in 2024-03-24T12:24:28.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:27 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:24:28.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:27 smithi006 ceph-mon[30108]: osdmap e670: 8 total, 8 up, 8 in 2024-03-24T12:24:28.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:27 smithi006 ceph-mon[30108]: pgmap v1034: 236 pgs: 236 active+clean; 582 KiB data, 713 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-24T12:24:28.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:27 smithi006 ceph-mon[30108]: osdmap e671: 8 total, 8 up, 8 in 2024-03-24T12:24:28.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:27 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:24:28.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:27 smithi006 ceph-mon[41734]: osdmap e670: 8 total, 8 up, 8 in 2024-03-24T12:24:28.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:27 smithi006 ceph-mon[41734]: pgmap v1034: 236 pgs: 236 active+clean; 582 KiB data, 713 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-24T12:24:28.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:27 smithi006 ceph-mon[41734]: osdmap e671: 8 total, 8 up, 8 in 2024-03-24T12:24:28.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:27 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:24:29.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:29 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:24:29.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:29 smithi090 ceph-mon[33646]: osdmap e672: 8 total, 8 up, 8 in 2024-03-24T12:24:29.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:29 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:24:29.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:29 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:24:29.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:29 smithi006 ceph-mon[30108]: osdmap e672: 8 total, 8 up, 8 in 2024-03-24T12:24:29.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:29 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:24:29.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:29 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:24:29.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:29 smithi006 ceph-mon[41734]: osdmap e672: 8 total, 8 up, 8 in 2024-03-24T12:24:29.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:29 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:24:30.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:30 smithi090 ceph-mon[33646]: pgmap v1037: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 713 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-24T12:24:30.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:30 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-134", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:24:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:30 smithi090 ceph-mon[33646]: osdmap e673: 8 total, 8 up, 8 in 2024-03-24T12:24:30.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:30 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-134"}]: dispatch 2024-03-24T12:24:30.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:30 smithi006 ceph-mon[30108]: pgmap v1037: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 713 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-24T12:24:30.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:30 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-134", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:24:30.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:30 smithi006 ceph-mon[30108]: osdmap e673: 8 total, 8 up, 8 in 2024-03-24T12:24:30.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:30 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-134"}]: dispatch 2024-03-24T12:24:30.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:30 smithi006 ceph-mon[41734]: pgmap v1037: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 713 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-24T12:24:30.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:30 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-134", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:24:30.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:30 smithi006 ceph-mon[41734]: osdmap e673: 8 total, 8 up, 8 in 2024-03-24T12:24:30.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:30 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-134"}]: dispatch 2024-03-24T12:24:31.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-134"}]': finished 2024-03-24T12:24:31.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:31 smithi090 ceph-mon[33646]: osdmap e674: 8 total, 8 up, 8 in 2024-03-24T12:24:31.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-134", "mode": "writeback"}]: dispatch 2024-03-24T12:24:31.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:31 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:24:31.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-134", "mode": "writeback"}]': finished 2024-03-24T12:24:31.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:31 smithi090 ceph-mon[33646]: osdmap e675: 8 total, 8 up, 8 in 2024-03-24T12:24:31.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:24:31.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-134"}]': finished 2024-03-24T12:24:31.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:31 smithi006 ceph-mon[30108]: osdmap e674: 8 total, 8 up, 8 in 2024-03-24T12:24:31.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-134", "mode": "writeback"}]: dispatch 2024-03-24T12:24:31.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:31 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:24:31.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-134", "mode": "writeback"}]': finished 2024-03-24T12:24:31.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:31 smithi006 ceph-mon[30108]: osdmap e675: 8 total, 8 up, 8 in 2024-03-24T12:24:31.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:24:31.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-134"}]': finished 2024-03-24T12:24:31.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:31 smithi006 ceph-mon[41734]: osdmap e674: 8 total, 8 up, 8 in 2024-03-24T12:24:31.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-134", "mode": "writeback"}]: dispatch 2024-03-24T12:24:31.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:31 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:24:31.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-134", "mode": "writeback"}]': finished 2024-03-24T12:24:31.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:31 smithi006 ceph-mon[41734]: osdmap e675: 8 total, 8 up, 8 in 2024-03-24T12:24:31.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:24:32.817 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:32 smithi090 ceph-mon[33646]: pgmap v1041: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 713 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:24:32.817 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:32 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:24:32.817 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:32 smithi090 ceph-mon[33646]: osdmap e676: 8 total, 8 up, 8 in 2024-03-24T12:24:32.817 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:32 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-134"}]: dispatch 2024-03-24T12:24:32.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:32 smithi006 ceph-mon[30108]: pgmap v1041: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 713 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:24:32.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:32 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:24:32.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:32 smithi006 ceph-mon[30108]: osdmap e676: 8 total, 8 up, 8 in 2024-03-24T12:24:32.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:32 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-134"}]: dispatch 2024-03-24T12:24:32.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:32 smithi006 ceph-mon[41734]: pgmap v1041: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 713 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:24:32.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:32 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:24:32.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:32 smithi006 ceph-mon[41734]: osdmap e676: 8 total, 8 up, 8 in 2024-03-24T12:24:32.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:32 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-134"}]: dispatch 2024-03-24T12:24:33.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:24:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:24:33.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:33 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:24:33.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:33 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-134"}]': finished 2024-03-24T12:24:33.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:33 smithi090 ceph-mon[33646]: osdmap e677: 8 total, 8 up, 8 in 2024-03-24T12:24:33.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:33 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:24:33.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:33 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-134"}]': finished 2024-03-24T12:24:33.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:33 smithi006 ceph-mon[30108]: osdmap e677: 8 total, 8 up, 8 in 2024-03-24T12:24:33.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:33 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:24:33.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:33 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-134"}]': finished 2024-03-24T12:24:33.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:33 smithi006 ceph-mon[41734]: osdmap e677: 8 total, 8 up, 8 in 2024-03-24T12:24:34.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:34 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:24:34.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:34 smithi090 ceph-mon[33646]: pgmap v1044: 268 pgs: 268 active+clean; 582 KiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-03-24T12:24:34.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:34 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:24:34.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:24:34.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:34 smithi090 ceph-mon[33646]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:34.928 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:34 smithi090 ceph-mon[33646]: osdmap e678: 8 total, 8 up, 8 in 2024-03-24T12:24:34.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:34 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:24:34.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:34 smithi006 ceph-mon[30108]: pgmap v1044: 268 pgs: 268 active+clean; 582 KiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-03-24T12:24:34.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:34 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:24:34.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:24:34.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:34 smithi006 ceph-mon[30108]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:34.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:34 smithi006 ceph-mon[30108]: osdmap e678: 8 total, 8 up, 8 in 2024-03-24T12:24:34.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:34 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:24:34.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:34 smithi006 ceph-mon[41734]: pgmap v1044: 268 pgs: 268 active+clean; 582 KiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-03-24T12:24:34.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:34 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:24:34.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:24:34.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:34 smithi006 ceph-mon[41734]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:34.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:34 smithi006 ceph-mon[41734]: osdmap e678: 8 total, 8 up, 8 in 2024-03-24T12:24:35.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:24:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:24:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:24:36.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:35 smithi090 ceph-mon[33646]: osdmap e679: 8 total, 8 up, 8 in 2024-03-24T12:24:36.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:35 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:24:36.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:35 smithi090 ceph-mon[33646]: pgmap v1047: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-24T12:24:36.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:35 smithi006 ceph-mon[30108]: osdmap e679: 8 total, 8 up, 8 in 2024-03-24T12:24:36.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:35 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:24:36.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:35 smithi006 ceph-mon[30108]: pgmap v1047: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-24T12:24:36.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:35 smithi006 ceph-mon[41734]: osdmap e679: 8 total, 8 up, 8 in 2024-03-24T12:24:36.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:35 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:24:36.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:35 smithi006 ceph-mon[41734]: pgmap v1047: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-24T12:24:37.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:36 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:24:37.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:36 smithi090 ceph-mon[33646]: osdmap e680: 8 total, 8 up, 8 in 2024-03-24T12:24:37.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:36 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:24:37.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:36 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:24:37.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:36 smithi006 ceph-mon[30108]: osdmap e680: 8 total, 8 up, 8 in 2024-03-24T12:24:37.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:36 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:24:37.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:36 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:24:37.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:36 smithi006 ceph-mon[41734]: osdmap e680: 8 total, 8 up, 8 in 2024-03-24T12:24:37.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:36 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:24:38.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-136", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:24:38.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:37 smithi090 ceph-mon[33646]: osdmap e681: 8 total, 8 up, 8 in 2024-03-24T12:24:38.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-136"}]: dispatch 2024-03-24T12:24:38.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:37 smithi090 ceph-mon[33646]: pgmap v1050: 268 pgs: 268 active+clean; 582 KiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:38.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-136", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:24:38.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:37 smithi006 ceph-mon[41734]: osdmap e681: 8 total, 8 up, 8 in 2024-03-24T12:24:38.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-136"}]: dispatch 2024-03-24T12:24:38.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:37 smithi006 ceph-mon[41734]: pgmap v1050: 268 pgs: 268 active+clean; 582 KiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:38.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-136", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:24:38.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:37 smithi006 ceph-mon[30108]: osdmap e681: 8 total, 8 up, 8 in 2024-03-24T12:24:38.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-136"}]: dispatch 2024-03-24T12:24:38.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:37 smithi006 ceph-mon[30108]: pgmap v1050: 268 pgs: 268 active+clean; 582 KiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:39.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:39 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-136"}]': finished 2024-03-24T12:24:39.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:39 smithi090 ceph-mon[33646]: osdmap e682: 8 total, 8 up, 8 in 2024-03-24T12:24:39.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:39 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-136", "mode": "writeback"}]: dispatch 2024-03-24T12:24:39.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-136"}]': finished 2024-03-24T12:24:39.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:39 smithi006 ceph-mon[30108]: osdmap e682: 8 total, 8 up, 8 in 2024-03-24T12:24:39.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:39 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-136", "mode": "writeback"}]: dispatch 2024-03-24T12:24:39.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:39 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-136"}]': finished 2024-03-24T12:24:39.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:39 smithi006 ceph-mon[41734]: osdmap e682: 8 total, 8 up, 8 in 2024-03-24T12:24:39.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:39 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-136", "mode": "writeback"}]: dispatch 2024-03-24T12:24:40.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:40 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:24:40.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-136", "mode": "writeback"}]': finished 2024-03-24T12:24:40.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:40 smithi090 ceph-mon[33646]: osdmap e683: 8 total, 8 up, 8 in 2024-03-24T12:24:40.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:40 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:24:40.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:40 smithi090 ceph-mon[33646]: pgmap v1053: 268 pgs: 268 active+clean; 582 KiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:40.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:40 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:24:40.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-136", "mode": "writeback"}]': finished 2024-03-24T12:24:40.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:40 smithi006 ceph-mon[30108]: osdmap e683: 8 total, 8 up, 8 in 2024-03-24T12:24:40.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:40 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:24:40.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:40 smithi006 ceph-mon[30108]: pgmap v1053: 268 pgs: 268 active+clean; 582 KiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:40.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:40 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:24:40.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-136", "mode": "writeback"}]': finished 2024-03-24T12:24:40.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:40 smithi006 ceph-mon[41734]: osdmap e683: 8 total, 8 up, 8 in 2024-03-24T12:24:40.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:40 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:24:40.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:40 smithi006 ceph-mon[41734]: pgmap v1053: 268 pgs: 268 active+clean; 582 KiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:41.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:24:41.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:41 smithi090 ceph-mon[33646]: osdmap e684: 8 total, 8 up, 8 in 2024-03-24T12:24:41.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-136"}]: dispatch 2024-03-24T12:24:41.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:41 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:41.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:24:41.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:41 smithi006 ceph-mon[30108]: osdmap e684: 8 total, 8 up, 8 in 2024-03-24T12:24:41.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-136"}]: dispatch 2024-03-24T12:24:41.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:41 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:41.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:24:41.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:41 smithi006 ceph-mon[41734]: osdmap e684: 8 total, 8 up, 8 in 2024-03-24T12:24:41.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-136"}]: dispatch 2024-03-24T12:24:41.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:41 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:41.579 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] c 2024-03-24T12:24:41.579 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.580 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.580 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.580 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.580 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.580 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.580 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.580 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.580 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.580 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.580 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.580 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.580 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.580 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.581 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.581 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.581 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.581 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.581 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.581 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.581 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.581 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.581 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.581 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.581 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.581 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.581 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.581 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.582 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.582 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.582 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.582 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.582 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.583 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.583 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.583 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.583 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.583 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.583 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.583 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.583 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.583 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.583 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.583 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.584 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.584 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.584 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.584 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.584 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.584 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.584 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.584 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.584 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.584 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.584 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.584 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.584 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.584 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.584 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.585 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.585 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.585 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.585 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.585 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.585 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.585 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.585 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.585 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.585 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.585 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.585 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.585 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.585 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.586 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.586 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.586 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.586 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.586 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.586 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.586 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.586 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.586 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.586 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.586 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.586 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.586 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.586 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.586 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.587 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.587 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.587 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.587 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.587 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.587 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.587 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.587 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.587 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.587 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:41.587 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:24:42.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:42 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:24:42.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:42 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-136"}]': finished 2024-03-24T12:24:42.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:42 smithi006 ceph-mon[30108]: osdmap e685: 8 total, 8 up, 8 in 2024-03-24T12:24:42.361 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:42 smithi006 ceph-mon[30108]: pgmap v1056: 268 pgs: 268 active+clean; 582 KiB data, 732 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:24:42.362 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:42 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:24:42.362 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:42 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-136"}]': finished 2024-03-24T12:24:42.362 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:42 smithi006 ceph-mon[41734]: osdmap e685: 8 total, 8 up, 8 in 2024-03-24T12:24:42.362 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:42 smithi006 ceph-mon[41734]: pgmap v1056: 268 pgs: 268 active+clean; 582 KiB data, 732 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:24:42.431 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:42 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:24:42.431 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:42 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-136"}]': finished 2024-03-24T12:24:42.431 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:42 smithi090 ceph-mon[33646]: osdmap e685: 8 total, 8 up, 8 in 2024-03-24T12:24:42.431 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:42 smithi090 ceph-mon[33646]: pgmap v1056: 268 pgs: 268 active+clean; 582 KiB data, 732 MiB used, 715 GiB / 715 GiB avail 2024-03-24T12:24:43.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:43 smithi090 ceph-mon[33646]: osdmap e686: 8 total, 8 up, 8 in 2024-03-24T12:24:43.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:24:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:24:43.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:43 smithi006 ceph-mon[30108]: osdmap e686: 8 total, 8 up, 8 in 2024-03-24T12:24:43.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:43 smithi006 ceph-mon[41734]: osdmap e686: 8 total, 8 up, 8 in 2024-03-24T12:24:44.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:44 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:24:44.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:44 smithi090 ceph-mon[33646]: osdmap e687: 8 total, 8 up, 8 in 2024-03-24T12:24:44.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:44 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:24:44.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:44 smithi090 ceph-mon[33646]: pgmap v1059: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-03-24T12:24:44.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:44 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:24:44.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:44 smithi006 ceph-mon[41734]: osdmap e687: 8 total, 8 up, 8 in 2024-03-24T12:24:44.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:44 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:24:44.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:44 smithi006 ceph-mon[41734]: pgmap v1059: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-03-24T12:24:44.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:44 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:24:44.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:44 smithi006 ceph-mon[30108]: osdmap e687: 8 total, 8 up, 8 in 2024-03-24T12:24:44.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:44 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:24:44.462 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:44 smithi006 ceph-mon[30108]: pgmap v1059: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-03-24T12:24:45.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:24:45.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:45 smithi090 ceph-mon[33646]: osdmap e688: 8 total, 8 up, 8 in 2024-03-24T12:24:45.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:24:45.460 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:24:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:24:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:24:45.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:24:45.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:45 smithi006 ceph-mon[30108]: osdmap e688: 8 total, 8 up, 8 in 2024-03-24T12:24:45.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:24:45.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:24:45.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:45 smithi006 ceph-mon[41734]: osdmap e688: 8 total, 8 up, 8 in 2024-03-24T12:24:45.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:24:46.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:46 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-138", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:24:46.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:46 smithi090 ceph-mon[33646]: osdmap e689: 8 total, 8 up, 8 in 2024-03-24T12:24:46.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:46 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-138","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:24:46.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:46 smithi090 ceph-mon[33646]: pgmap v1062: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-24T12:24:46.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:46 smithi090 ceph-mon[33646]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:46.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:46 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-138", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:24:46.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:46 smithi006 ceph-mon[30108]: osdmap e689: 8 total, 8 up, 8 in 2024-03-24T12:24:46.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:46 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-138","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:24:46.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:46 smithi006 ceph-mon[30108]: pgmap v1062: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-24T12:24:46.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:46 smithi006 ceph-mon[30108]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:46.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:46 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-138", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:24:46.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:46 smithi006 ceph-mon[41734]: osdmap e689: 8 total, 8 up, 8 in 2024-03-24T12:24:46.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:46 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-138","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:24:46.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:46 smithi006 ceph-mon[41734]: pgmap v1062: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-24T12:24:46.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:46 smithi006 ceph-mon[41734]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:47.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-138","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:24:47.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:47 smithi090 ceph-mon[33646]: osdmap e690: 8 total, 8 up, 8 in 2024-03-24T12:24:47.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-138","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:24:47.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-138","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:24:47.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:47 smithi006 ceph-mon[41734]: osdmap e690: 8 total, 8 up, 8 in 2024-03-24T12:24:47.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-138","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:24:47.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-138","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:24:47.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:47 smithi006 ceph-mon[30108]: osdmap e690: 8 total, 8 up, 8 in 2024-03-24T12:24:47.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-138","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:24:48.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-138","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:24:48.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:48 smithi090 ceph-mon[33646]: osdmap e691: 8 total, 8 up, 8 in 2024-03-24T12:24:48.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-24T12:24:48.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:48 smithi090 ceph-mon[33646]: pgmap v1065: 268 pgs: 268 active+clean; 4.4 MiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:48.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-24T12:24:48.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:48 smithi090 ceph-mon[33646]: osdmap e692: 8 total, 8 up, 8 in 2024-03-24T12:24:48.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-138","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:24:48.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:48 smithi006 ceph-mon[30108]: osdmap e691: 8 total, 8 up, 8 in 2024-03-24T12:24:48.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-24T12:24:48.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:48 smithi006 ceph-mon[30108]: pgmap v1065: 268 pgs: 268 active+clean; 4.4 MiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:48.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-24T12:24:48.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:48 smithi006 ceph-mon[30108]: osdmap e692: 8 total, 8 up, 8 in 2024-03-24T12:24:48.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-138","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:24:48.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:48 smithi006 ceph-mon[41734]: osdmap e691: 8 total, 8 up, 8 in 2024-03-24T12:24:48.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-24T12:24:48.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:48 smithi006 ceph-mon[41734]: pgmap v1065: 268 pgs: 268 active+clean; 4.4 MiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:48.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-24T12:24:48.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:48 smithi006 ceph-mon[41734]: osdmap e692: 8 total, 8 up, 8 in 2024-03-24T12:24:49.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:49 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:24:49.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:49 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:24:49.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:49 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:24:50.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:50 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:24:50.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:50 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-138"}]: dispatch 2024-03-24T12:24:50.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:50 smithi090 ceph-mon[33646]: pgmap v1067: 268 pgs: 268 active+clean; 4.4 MiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:50.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:50 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:24:50.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:50 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-138"}]: dispatch 2024-03-24T12:24:50.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:50 smithi006 ceph-mon[30108]: pgmap v1067: 268 pgs: 268 active+clean; 4.4 MiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:50.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:50 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:24:50.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:50 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-138"}]: dispatch 2024-03-24T12:24:50.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:50 smithi006 ceph-mon[41734]: pgmap v1067: 268 pgs: 268 active+clean; 4.4 MiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:51.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:51 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-138"}]': finished 2024-03-24T12:24:51.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:51 smithi090 ceph-mon[33646]: osdmap e693: 8 total, 8 up, 8 in 2024-03-24T12:24:51.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:51 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:51.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-138"}]': finished 2024-03-24T12:24:51.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:51 smithi006 ceph-mon[30108]: osdmap e693: 8 total, 8 up, 8 in 2024-03-24T12:24:51.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:51 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:51.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:51 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-138"}]': finished 2024-03-24T12:24:51.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:51 smithi006 ceph-mon[41734]: osdmap e693: 8 total, 8 up, 8 in 2024-03-24T12:24:51.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:51 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:24:52.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:52 smithi090 ceph-mon[33646]: osdmap e694: 8 total, 8 up, 8 in 2024-03-24T12:24:52.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:52 smithi090 ceph-mon[33646]: pgmap v1070: 236 pgs: 236 active+clean; 4.4 MiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:52.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:52 smithi090 ceph-mon[33646]: osdmap e695: 8 total, 8 up, 8 in 2024-03-24T12:24:52.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:52 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:24:52.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:52 smithi006 ceph-mon[30108]: osdmap e694: 8 total, 8 up, 8 in 2024-03-24T12:24:52.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:52 smithi006 ceph-mon[30108]: pgmap v1070: 236 pgs: 236 active+clean; 4.4 MiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:52.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:52 smithi006 ceph-mon[30108]: osdmap e695: 8 total, 8 up, 8 in 2024-03-24T12:24:52.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:52 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:24:52.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:52 smithi006 ceph-mon[41734]: osdmap e694: 8 total, 8 up, 8 in 2024-03-24T12:24:52.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:52 smithi006 ceph-mon[41734]: pgmap v1070: 236 pgs: 236 active+clean; 4.4 MiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:52.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:52 smithi006 ceph-mon[41734]: osdmap e695: 8 total, 8 up, 8 in 2024-03-24T12:24:52.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:52 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:24:53.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:24:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:24:54.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:54 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:24:54.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:24:54.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:54 smithi090 ceph-mon[33646]: osdmap e696: 8 total, 8 up, 8 in 2024-03-24T12:24:54.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:24:54.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:54 smithi090 ceph-mon[33646]: pgmap v1073: 268 pgs: 32 creating+peering, 236 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:54.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:54 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:24:54.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:24:54.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:54 smithi006 ceph-mon[30108]: osdmap e696: 8 total, 8 up, 8 in 2024-03-24T12:24:54.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:24:54.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:54 smithi006 ceph-mon[30108]: pgmap v1073: 268 pgs: 32 creating+peering, 236 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:54.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:54 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:24:54.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:24:54.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:54 smithi006 ceph-mon[41734]: osdmap e696: 8 total, 8 up, 8 in 2024-03-24T12:24:54.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:24:54.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:54 smithi006 ceph-mon[41734]: pgmap v1073: 268 pgs: 32 creating+peering, 236 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:55.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:55 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-140", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:24:55.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:55 smithi090 ceph-mon[33646]: osdmap e697: 8 total, 8 up, 8 in 2024-03-24T12:24:55.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:55 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_count","val": "3"}]: dispatch 2024-03-24T12:24:55.460 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:24:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:24:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:24:55.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:55 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-140", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:24:55.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:55 smithi006 ceph-mon[30108]: osdmap e697: 8 total, 8 up, 8 in 2024-03-24T12:24:55.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:55 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_count","val": "3"}]: dispatch 2024-03-24T12:24:55.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:55 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-140", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:24:55.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:55 smithi006 ceph-mon[41734]: osdmap e697: 8 total, 8 up, 8 in 2024-03-24T12:24:55.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:55 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_count","val": "3"}]: dispatch 2024-03-24T12:24:56.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:56 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_count","val": "3"}]': finished 2024-03-24T12:24:56.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:56 smithi090 ceph-mon[33646]: osdmap e698: 8 total, 8 up, 8 in 2024-03-24T12:24:56.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:56 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_period","val": "3"}]: dispatch 2024-03-24T12:24:56.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:56 smithi090 ceph-mon[33646]: pgmap v1076: 268 pgs: 32 creating+peering, 236 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:56.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:56 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_count","val": "3"}]': finished 2024-03-24T12:24:56.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:56 smithi006 ceph-mon[30108]: osdmap e698: 8 total, 8 up, 8 in 2024-03-24T12:24:56.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:56 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_period","val": "3"}]: dispatch 2024-03-24T12:24:56.461 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:56 smithi006 ceph-mon[30108]: pgmap v1076: 268 pgs: 32 creating+peering, 236 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:56.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:56 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_count","val": "3"}]': finished 2024-03-24T12:24:56.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:56 smithi006 ceph-mon[41734]: osdmap e698: 8 total, 8 up, 8 in 2024-03-24T12:24:56.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:56 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_period","val": "3"}]: dispatch 2024-03-24T12:24:56.462 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:56 smithi006 ceph-mon[41734]: pgmap v1076: 268 pgs: 32 creating+peering, 236 active+clean; 4.4 MiB data, 743 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:57.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:57 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_period","val": "3"}]': finished 2024-03-24T12:24:57.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:57 smithi090 ceph-mon[33646]: osdmap e699: 8 total, 8 up, 8 in 2024-03-24T12:24:57.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:57 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:24:57.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:57 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_period","val": "3"}]': finished 2024-03-24T12:24:57.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:57 smithi006 ceph-mon[30108]: osdmap e699: 8 total, 8 up, 8 in 2024-03-24T12:24:57.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:57 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:24:57.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:57 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_period","val": "3"}]': finished 2024-03-24T12:24:57.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:57 smithi006 ceph-mon[41734]: osdmap e699: 8 total, 8 up, 8 in 2024-03-24T12:24:57.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:57 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:24:58.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:58 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_type","val": "bloom"}]': finished 2024-03-24T12:24:58.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:58 smithi090 ceph-mon[33646]: osdmap e700: 8 total, 8 up, 8 in 2024-03-24T12:24:58.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:58 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-24T12:24:58.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:58 smithi090 ceph-mon[33646]: pgmap v1079: 268 pgs: 268 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:58.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:58 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-24T12:24:58.428 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:24:58 smithi090 ceph-mon[33646]: osdmap e701: 8 total, 8 up, 8 in 2024-03-24T12:24:58.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:58 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_type","val": "bloom"}]': finished 2024-03-24T12:24:58.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:58 smithi006 ceph-mon[30108]: osdmap e700: 8 total, 8 up, 8 in 2024-03-24T12:24:58.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:58 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-24T12:24:58.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:58 smithi006 ceph-mon[30108]: pgmap v1079: 268 pgs: 268 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:58.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:58 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-24T12:24:58.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:24:58 smithi006 ceph-mon[30108]: osdmap e701: 8 total, 8 up, 8 in 2024-03-24T12:24:58.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:58 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_type","val": "bloom"}]': finished 2024-03-24T12:24:58.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:58 smithi006 ceph-mon[41734]: osdmap e700: 8 total, 8 up, 8 in 2024-03-24T12:24:58.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:58 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-24T12:24:58.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:58 smithi006 ceph-mon[41734]: pgmap v1079: 268 pgs: 268 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:24:58.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:58 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-24T12:24:58.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:24:58 smithi006 ceph-mon[41734]: osdmap e701: 8 total, 8 up, 8 in 2024-03-24T12:25:00.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:00 smithi090 ceph-mon[33646]: pgmap v1081: 268 pgs: 268 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:25:00.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:00 smithi006 ceph-mon[30108]: pgmap v1081: 268 pgs: 268 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:25:00.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:00 smithi006 ceph-mon[41734]: pgmap v1081: 268 pgs: 268 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:25:02.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:02 smithi090 ceph-mon[33646]: pgmap v1082: 268 pgs: 268 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:25:02.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:02 smithi006 ceph-mon[30108]: pgmap v1082: 268 pgs: 268 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:25:02.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:02 smithi006 ceph-mon[41734]: pgmap v1082: 268 pgs: 268 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:25:03.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:25:02 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:25:04.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:04 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:25:04.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:04 smithi090 ceph-mon[33646]: pgmap v1083: 268 pgs: 268 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 5.7 KiB/s wr, 2 op/s 2024-03-24T12:25:04.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:25:04.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:04 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:25:04.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:04 smithi006 ceph-mon[30108]: pgmap v1083: 268 pgs: 268 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 5.7 KiB/s wr, 2 op/s 2024-03-24T12:25:04.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:25:04.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:04 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:25:04.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:04 smithi006 ceph-mon[41734]: pgmap v1083: 268 pgs: 268 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 5.7 KiB/s wr, 2 op/s 2024-03-24T12:25:04.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:25:05.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:25:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:25:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:25:06.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:06 smithi090 ceph-mon[33646]: pgmap v1084: 268 pgs: 268 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 1 op/s 2024-03-24T12:25:06.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:06 smithi006 ceph-mon[30108]: pgmap v1084: 268 pgs: 268 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 1 op/s 2024-03-24T12:25:06.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:06 smithi006 ceph-mon[41734]: pgmap v1084: 268 pgs: 268 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 1 op/s 2024-03-24T12:25:07.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:07 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:25:07.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:07 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:25:07.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:07 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:25:07.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:07 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:25:07.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:07 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:25:07.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:07 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:25:07.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:07 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:25:07.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:07 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:25:07.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:07 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:25:08.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:08 smithi090 ceph-mon[33646]: pgmap v1085: 268 pgs: 268 active+clean; 4.5 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-03-24T12:25:08.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:08 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:25:08.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:08 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:25:08.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:08 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:25:08.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:08 smithi006 ceph-mon[30108]: pgmap v1085: 268 pgs: 268 active+clean; 4.5 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-03-24T12:25:08.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:08 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:25:08.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:08 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:25:08.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:08 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:25:08.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:08 smithi006 ceph-mon[41734]: pgmap v1085: 268 pgs: 268 active+clean; 4.5 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-03-24T12:25:08.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:08 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:25:08.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:08 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:25:08.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:08 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:25:10.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:10 smithi090 ceph-mon[33646]: pgmap v1086: 268 pgs: 268 active+clean; 4.5 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 916 B/s rd, 7.4 KiB/s wr, 2 op/s 2024-03-24T12:25:10.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:25:10.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-140"}]: dispatch 2024-03-24T12:25:10.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:10 smithi006 ceph-mon[30108]: pgmap v1086: 268 pgs: 268 active+clean; 4.5 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 916 B/s rd, 7.4 KiB/s wr, 2 op/s 2024-03-24T12:25:10.800 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:25:10.800 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-140"}]: dispatch 2024-03-24T12:25:10.800 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:10 smithi006 ceph-mon[41734]: pgmap v1086: 268 pgs: 268 active+clean; 4.5 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 916 B/s rd, 7.4 KiB/s wr, 2 op/s 2024-03-24T12:25:10.800 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:25:10.801 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-140"}]: dispatch 2024-03-24T12:25:11.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-140"}]': finished 2024-03-24T12:25:11.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:11 smithi090 ceph-mon[33646]: osdmap e702: 8 total, 8 up, 8 in 2024-03-24T12:25:11.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-140"}]': finished 2024-03-24T12:25:11.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:11 smithi006 ceph-mon[30108]: osdmap e702: 8 total, 8 up, 8 in 2024-03-24T12:25:11.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-140"}]': finished 2024-03-24T12:25:11.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:11 smithi006 ceph-mon[41734]: osdmap e702: 8 total, 8 up, 8 in 2024-03-24T12:25:12.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:12 smithi090 ceph-mon[33646]: pgmap v1088: 268 pgs: 268 active+clean; 4.5 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-03-24T12:25:12.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:12 smithi090 ceph-mon[33646]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:25:12.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:12 smithi090 ceph-mon[33646]: osdmap e703: 8 total, 8 up, 8 in 2024-03-24T12:25:12.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:12 smithi006 ceph-mon[30108]: pgmap v1088: 268 pgs: 268 active+clean; 4.5 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-03-24T12:25:12.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:12 smithi006 ceph-mon[30108]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:25:12.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:12 smithi006 ceph-mon[30108]: osdmap e703: 8 total, 8 up, 8 in 2024-03-24T12:25:12.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:12 smithi006 ceph-mon[41734]: pgmap v1088: 268 pgs: 268 active+clean; 4.5 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-03-24T12:25:12.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:12 smithi006 ceph-mon[41734]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:25:12.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:12 smithi006 ceph-mon[41734]: osdmap e703: 8 total, 8 up, 8 in 2024-03-24T12:25:13.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:25:12 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:25:13.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:13 smithi090 ceph-mon[33646]: osdmap e704: 8 total, 8 up, 8 in 2024-03-24T12:25:13.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:13 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:25:13.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:13 smithi006 ceph-mon[30108]: osdmap e704: 8 total, 8 up, 8 in 2024-03-24T12:25:13.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:13 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:25:13.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:13 smithi006 ceph-mon[41734]: osdmap e704: 8 total, 8 up, 8 in 2024-03-24T12:25:13.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:13 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:25:14.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:14 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:25:14.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:14 smithi090 ceph-mon[33646]: pgmap v1091: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:25:14.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:25:14.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:14 smithi090 ceph-mon[33646]: osdmap e705: 8 total, 8 up, 8 in 2024-03-24T12:25:14.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:14 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:25:14.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:14 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:25:14.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:14 smithi006 ceph-mon[30108]: pgmap v1091: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:25:14.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:25:14.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:14 smithi006 ceph-mon[30108]: osdmap e705: 8 total, 8 up, 8 in 2024-03-24T12:25:14.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:14 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:25:14.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:14 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:25:14.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:14 smithi006 ceph-mon[41734]: pgmap v1091: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:25:14.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:25:14.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:14 smithi006 ceph-mon[41734]: osdmap e705: 8 total, 8 up, 8 in 2024-03-24T12:25:14.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:14 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:25:15.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-142", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:25:15.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:15 smithi090 ceph-mon[33646]: osdmap e706: 8 total, 8 up, 8 in 2024-03-24T12:25:15.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-142"}]: dispatch 2024-03-24T12:25:15.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-142", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:25:15.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:15 smithi006 ceph-mon[30108]: osdmap e706: 8 total, 8 up, 8 in 2024-03-24T12:25:15.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-142"}]: dispatch 2024-03-24T12:25:15.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:25:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:25:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:25:15.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-142", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:25:15.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:15 smithi006 ceph-mon[41734]: osdmap e706: 8 total, 8 up, 8 in 2024-03-24T12:25:15.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-142"}]: dispatch 2024-03-24T12:25:16.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:16 smithi090 ceph-mon[33646]: pgmap v1094: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:25:16.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:16 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-142"}]': finished 2024-03-24T12:25:16.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:16 smithi090 ceph-mon[33646]: osdmap e707: 8 total, 8 up, 8 in 2024-03-24T12:25:16.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:16 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-142", "mode": "writeback"}]: dispatch 2024-03-24T12:25:16.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:16 smithi006 ceph-mon[30108]: pgmap v1094: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:25:16.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:16 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-142"}]': finished 2024-03-24T12:25:16.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:16 smithi006 ceph-mon[30108]: osdmap e707: 8 total, 8 up, 8 in 2024-03-24T12:25:16.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:16 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-142", "mode": "writeback"}]: dispatch 2024-03-24T12:25:16.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:16 smithi006 ceph-mon[41734]: pgmap v1094: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:25:16.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:16 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-142"}]': finished 2024-03-24T12:25:16.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:16 smithi006 ceph-mon[41734]: osdmap e707: 8 total, 8 up, 8 in 2024-03-24T12:25:16.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:16 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-142", "mode": "writeback"}]: dispatch 2024-03-24T12:25:17.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:17 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:25:17.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-142", "mode": "writeback"}]': finished 2024-03-24T12:25:17.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:17 smithi090 ceph-mon[33646]: osdmap e708: 8 total, 8 up, 8 in 2024-03-24T12:25:17.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:25:17.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:17 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:25:17.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-142", "mode": "writeback"}]': finished 2024-03-24T12:25:17.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:17 smithi006 ceph-mon[41734]: osdmap e708: 8 total, 8 up, 8 in 2024-03-24T12:25:17.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:25:17.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:17 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:25:17.713 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-142", "mode": "writeback"}]': finished 2024-03-24T12:25:17.714 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:17 smithi006 ceph-mon[30108]: osdmap e708: 8 total, 8 up, 8 in 2024-03-24T12:25:17.714 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:25:18.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:18 smithi006 ceph-mon[30108]: pgmap v1097: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-24T12:25:18.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:25:18.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:18 smithi006 ceph-mon[30108]: osdmap e709: 8 total, 8 up, 8 in 2024-03-24T12:25:18.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:25:18.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:25:18.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:18 smithi006 ceph-mon[41734]: pgmap v1097: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-24T12:25:18.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:25:18.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:18 smithi006 ceph-mon[41734]: osdmap e709: 8 total, 8 up, 8 in 2024-03-24T12:25:18.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:25:18.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:25:18.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:18 smithi090 ceph-mon[33646]: pgmap v1097: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-24T12:25:18.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:25:18.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:18 smithi090 ceph-mon[33646]: osdmap e709: 8 total, 8 up, 8 in 2024-03-24T12:25:18.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:25:18.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:25:19.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:19 smithi006 ceph-mon[30108]: osdmap e710: 8 total, 8 up, 8 in 2024-03-24T12:25:19.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:25:19.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:19 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:25:19.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:25:19.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:19 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:25:19.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_type","val": "bloom"}]': finished 2024-03-24T12:25:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:19 smithi006 ceph-mon[41734]: osdmap e710: 8 total, 8 up, 8 in 2024-03-24T12:25:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:25:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:19 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:25:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:25:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:19 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:25:19.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_type","val": "bloom"}]': finished 2024-03-24T12:25:19.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:19 smithi090 ceph-mon[33646]: osdmap e710: 8 total, 8 up, 8 in 2024-03-24T12:25:19.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:25:19.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:19 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:25:19.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:25:19.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:19 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:25:19.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_type","val": "bloom"}]': finished 2024-03-24T12:25:20.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:20 smithi006 ceph-mon[30108]: pgmap v1100: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-24T12:25:20.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:20 smithi006 ceph-mon[30108]: osdmap e711: 8 total, 8 up, 8 in 2024-03-24T12:25:20.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:20 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-24T12:25:20.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:20 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-24T12:25:20.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:20 smithi006 ceph-mon[30108]: osdmap e712: 8 total, 8 up, 8 in 2024-03-24T12:25:20.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:20 smithi006 ceph-mon[41734]: pgmap v1100: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-24T12:25:20.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:20 smithi006 ceph-mon[41734]: osdmap e711: 8 total, 8 up, 8 in 2024-03-24T12:25:20.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:20 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-24T12:25:20.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:20 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-24T12:25:20.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:20 smithi006 ceph-mon[41734]: osdmap e712: 8 total, 8 up, 8 in 2024-03-24T12:25:20.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:20 smithi090 ceph-mon[33646]: pgmap v1100: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-24T12:25:20.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:20 smithi090 ceph-mon[33646]: osdmap e711: 8 total, 8 up, 8 in 2024-03-24T12:25:20.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:20 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-24T12:25:20.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:20 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-24T12:25:20.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:20 smithi090 ceph-mon[33646]: osdmap e712: 8 total, 8 up, 8 in 2024-03-24T12:25:21.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-24T12:25:21.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:21 smithi006 ceph-mon[30108]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:25:21.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-24T12:25:21.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-24T12:25:21.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:21 smithi006 ceph-mon[41734]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:25:21.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-24T12:25:21.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-24T12:25:21.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:21 smithi090 ceph-mon[33646]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:25:21.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-24T12:25:22.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:22 smithi006 ceph-mon[30108]: pgmap v1103: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:25:22.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:22 smithi006 ceph-mon[30108]: osdmap e713: 8 total, 8 up, 8 in 2024-03-24T12:25:22.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-24T12:25:22.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-24T12:25:22.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:22 smithi006 ceph-mon[30108]: osdmap e714: 8 total, 8 up, 8 in 2024-03-24T12:25:22.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:22 smithi006 ceph-mon[41734]: pgmap v1103: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:25:22.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:22 smithi006 ceph-mon[41734]: osdmap e713: 8 total, 8 up, 8 in 2024-03-24T12:25:22.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-24T12:25:22.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-24T12:25:22.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:22 smithi006 ceph-mon[41734]: osdmap e714: 8 total, 8 up, 8 in 2024-03-24T12:25:22.854 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:22 smithi090 ceph-mon[33646]: pgmap v1103: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:25:22.854 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:22 smithi090 ceph-mon[33646]: osdmap e713: 8 total, 8 up, 8 in 2024-03-24T12:25:22.854 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-24T12:25:22.854 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-24T12:25:22.854 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:22 smithi090 ceph-mon[33646]: osdmap e714: 8 total, 8 up, 8 in 2024-03-24T12:25:23.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:25:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:25:24.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:24 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:25:24.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:24 smithi006 ceph-mon[30108]: pgmap v1106: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:25:24.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:25:24.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:24 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:25:24.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:24 smithi006 ceph-mon[41734]: pgmap v1106: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:25:24.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:25:24.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:24 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:25:24.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:24 smithi090 ceph-mon[33646]: pgmap v1106: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:25:24.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:25:25.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:25:25.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:25 smithi006 ceph-mon[30108]: osdmap e715: 8 total, 8 up, 8 in 2024-03-24T12:25:25.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:25 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-142"}]: dispatch 2024-03-24T12:25:25.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:25:25.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:25 smithi006 ceph-mon[41734]: osdmap e715: 8 total, 8 up, 8 in 2024-03-24T12:25:25.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:25 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-142"}]: dispatch 2024-03-24T12:25:25.711 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:25:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:25:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:25:25.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:25:25.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:25 smithi090 ceph-mon[33646]: osdmap e715: 8 total, 8 up, 8 in 2024-03-24T12:25:25.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:25 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-142"}]: dispatch 2024-03-24T12:25:27.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:26 smithi090 ceph-mon[33646]: pgmap v1108: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T12:25:27.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-142"}]': finished 2024-03-24T12:25:27.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:26 smithi090 ceph-mon[33646]: osdmap e716: 8 total, 8 up, 8 in 2024-03-24T12:25:27.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:25:27.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:26 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-142"}]: dispatch 2024-03-24T12:25:27.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:26 smithi006 ceph-mon[41734]: pgmap v1108: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T12:25:27.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-142"}]': finished 2024-03-24T12:25:27.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:26 smithi006 ceph-mon[41734]: osdmap e716: 8 total, 8 up, 8 in 2024-03-24T12:25:27.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:25:27.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:26 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-142"}]: dispatch 2024-03-24T12:25:27.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:26 smithi006 ceph-mon[30108]: pgmap v1108: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T12:25:27.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-142"}]': finished 2024-03-24T12:25:27.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:26 smithi006 ceph-mon[30108]: osdmap e716: 8 total, 8 up, 8 in 2024-03-24T12:25:27.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:25:27.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:26 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-142"}]: dispatch 2024-03-24T12:25:28.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:27 smithi090 ceph-mon[33646]: osdmap e717: 8 total, 8 up, 8 in 2024-03-24T12:25:28.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:27 smithi090 ceph-mon[33646]: osdmap e718: 8 total, 8 up, 8 in 2024-03-24T12:25:28.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:27 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:25:28.209 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:27 smithi006 ceph-mon[30108]: osdmap e717: 8 total, 8 up, 8 in 2024-03-24T12:25:28.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:27 smithi006 ceph-mon[30108]: osdmap e718: 8 total, 8 up, 8 in 2024-03-24T12:25:28.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:27 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:25:28.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:27 smithi006 ceph-mon[41734]: osdmap e717: 8 total, 8 up, 8 in 2024-03-24T12:25:28.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:27 smithi006 ceph-mon[41734]: osdmap e718: 8 total, 8 up, 8 in 2024-03-24T12:25:28.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:27 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:25:29.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:28 smithi090 ceph-mon[33646]: pgmap v1111: 236 pgs: 236 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-24T12:25:29.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:25:29.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:28 smithi090 ceph-mon[33646]: osdmap e719: 8 total, 8 up, 8 in 2024-03-24T12:25:29.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:28 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:25:29.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:28 smithi006 ceph-mon[30108]: pgmap v1111: 236 pgs: 236 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-24T12:25:29.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:25:29.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:28 smithi006 ceph-mon[30108]: osdmap e719: 8 total, 8 up, 8 in 2024-03-24T12:25:29.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:28 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:25:29.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:28 smithi006 ceph-mon[41734]: pgmap v1111: 236 pgs: 236 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-24T12:25:29.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:25:29.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:28 smithi006 ceph-mon[41734]: osdmap e719: 8 total, 8 up, 8 in 2024-03-24T12:25:29.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:28 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:25:30.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:30 smithi090 ceph-mon[33646]: pgmap v1114: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-24T12:25:30.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:30 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-144", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:25:30.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:30 smithi090 ceph-mon[33646]: osdmap e720: 8 total, 8 up, 8 in 2024-03-24T12:25:30.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:30 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-144"}]: dispatch 2024-03-24T12:25:30.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:30 smithi006 ceph-mon[30108]: pgmap v1114: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-24T12:25:30.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:30 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-144", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:25:30.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:30 smithi006 ceph-mon[30108]: osdmap e720: 8 total, 8 up, 8 in 2024-03-24T12:25:30.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:30 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-144"}]: dispatch 2024-03-24T12:25:30.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:30 smithi006 ceph-mon[41734]: pgmap v1114: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-24T12:25:30.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:30 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-144", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:25:30.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:30 smithi006 ceph-mon[41734]: osdmap e720: 8 total, 8 up, 8 in 2024-03-24T12:25:30.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:30 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-144"}]: dispatch 2024-03-24T12:25:31.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-144"}]': finished 2024-03-24T12:25:31.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:31 smithi090 ceph-mon[33646]: osdmap e721: 8 total, 8 up, 8 in 2024-03-24T12:25:31.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-144", "mode": "readproxy"}]: dispatch 2024-03-24T12:25:31.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:31 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:25:31.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:31 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-144", "mode": "readproxy"}]': finished 2024-03-24T12:25:31.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:31 smithi090 ceph-mon[33646]: osdmap e722: 8 total, 8 up, 8 in 2024-03-24T12:25:31.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-144"}]': finished 2024-03-24T12:25:31.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:31 smithi006 ceph-mon[30108]: osdmap e721: 8 total, 8 up, 8 in 2024-03-24T12:25:31.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-144", "mode": "readproxy"}]: dispatch 2024-03-24T12:25:31.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:31 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:25:31.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:31 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-144", "mode": "readproxy"}]': finished 2024-03-24T12:25:31.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:31 smithi006 ceph-mon[30108]: osdmap e722: 8 total, 8 up, 8 in 2024-03-24T12:25:31.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-144"}]': finished 2024-03-24T12:25:31.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:31 smithi006 ceph-mon[41734]: osdmap e721: 8 total, 8 up, 8 in 2024-03-24T12:25:31.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-144", "mode": "readproxy"}]: dispatch 2024-03-24T12:25:31.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:31 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:25:31.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:31 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-144", "mode": "readproxy"}]': finished 2024-03-24T12:25:31.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:31 smithi006 ceph-mon[41734]: osdmap e722: 8 total, 8 up, 8 in 2024-03-24T12:25:32.857 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:32 smithi090 ceph-mon[33646]: pgmap v1117: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:25:32.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:32 smithi006 ceph-mon[30108]: pgmap v1117: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:25:32.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:32 smithi006 ceph-mon[41734]: pgmap v1117: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:25:33.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:25:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:25:34.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:34 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:25:34.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:34 smithi090 ceph-mon[33646]: pgmap v1119: 268 pgs: 268 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 211 B/s wr, 1 op/s 2024-03-24T12:25:34.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:25:34.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:34 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:25:34.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:34 smithi006 ceph-mon[30108]: pgmap v1119: 268 pgs: 268 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 211 B/s wr, 1 op/s 2024-03-24T12:25:34.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:25:34.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:34 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:25:34.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:34 smithi006 ceph-mon[41734]: pgmap v1119: 268 pgs: 268 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 211 B/s wr, 1 op/s 2024-03-24T12:25:34.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:25:35.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:25:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:25:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:25:36.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:36 smithi090 ceph-mon[33646]: pgmap v1120: 268 pgs: 268 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-03-24T12:25:36.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:36 smithi006 ceph-mon[30108]: pgmap v1120: 268 pgs: 268 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-03-24T12:25:36.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:36 smithi006 ceph-mon[41734]: pgmap v1120: 268 pgs: 268 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-03-24T12:25:38.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:38 smithi090 ceph-mon[33646]: pgmap v1121: 268 pgs: 268 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 130 B/s wr, 1 op/s 2024-03-24T12:25:38.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:38 smithi006 ceph-mon[30108]: pgmap v1121: 268 pgs: 268 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 130 B/s wr, 1 op/s 2024-03-24T12:25:38.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:38 smithi006 ceph-mon[41734]: pgmap v1121: 268 pgs: 268 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 130 B/s wr, 1 op/s 2024-03-24T12:25:40.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:40 smithi090 ceph-mon[33646]: pgmap v1122: 268 pgs: 268 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 115 B/s wr, 1 op/s 2024-03-24T12:25:40.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:40 smithi006 ceph-mon[30108]: pgmap v1122: 268 pgs: 268 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 115 B/s wr, 1 op/s 2024-03-24T12:25:40.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:40 smithi006 ceph-mon[41734]: pgmap v1122: 268 pgs: 268 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 115 B/s wr, 1 op/s 2024-03-24T12:25:42.761 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:42 smithi006 ceph-mon[30108]: pgmap v1123: 268 pgs: 268 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 B/s wr, 1 op/s 2024-03-24T12:25:42.761 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:42 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:25:42.762 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:42 smithi006 ceph-mon[41734]: pgmap v1123: 268 pgs: 268 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 B/s wr, 1 op/s 2024-03-24T12:25:42.762 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:42 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:25:42.868 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:42 smithi090 ceph-mon[33646]: pgmap v1123: 268 pgs: 268 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 B/s wr, 1 op/s 2024-03-24T12:25:42.868 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:42 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:25:43.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:25:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:25:43.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:43 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:25:43.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:43 smithi090 ceph-mon[33646]: osdmap e723: 8 total, 8 up, 8 in 2024-03-24T12:25:43.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:43 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-144"}]: dispatch 2024-03-24T12:25:43.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:43 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:25:43.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:43 smithi006 ceph-mon[41734]: osdmap e723: 8 total, 8 up, 8 in 2024-03-24T12:25:43.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:43 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-144"}]: dispatch 2024-03-24T12:25:43.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:43 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:25:43.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:43 smithi006 ceph-mon[30108]: osdmap e723: 8 total, 8 up, 8 in 2024-03-24T12:25:43.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:43 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-144"}]: dispatch 2024-03-24T12:25:45.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:44 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:25:45.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:44 smithi090 ceph-mon[33646]: pgmap v1125: 268 pgs: 268 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:25:45.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:44 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:25:45.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:44 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-144"}]': finished 2024-03-24T12:25:45.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:44 smithi090 ceph-mon[33646]: osdmap e724: 8 total, 8 up, 8 in 2024-03-24T12:25:45.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:44 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:25:45.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:44 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-144"}]: dispatch 2024-03-24T12:25:45.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:44 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:25:45.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:44 smithi006 ceph-mon[30108]: pgmap v1125: 268 pgs: 268 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:25:45.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:44 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:25:45.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:44 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-144"}]': finished 2024-03-24T12:25:45.213 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:44 smithi006 ceph-mon[30108]: osdmap e724: 8 total, 8 up, 8 in 2024-03-24T12:25:45.214 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:44 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:25:45.214 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:44 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-144"}]: dispatch 2024-03-24T12:25:45.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:44 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:25:45.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:44 smithi006 ceph-mon[41734]: pgmap v1125: 268 pgs: 268 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:25:45.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:44 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:25:45.214 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:44 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-144"}]': finished 2024-03-24T12:25:45.215 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:44 smithi006 ceph-mon[41734]: osdmap e724: 8 total, 8 up, 8 in 2024-03-24T12:25:45.215 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:44 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:25:45.215 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:44 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-144"}]: dispatch 2024-03-24T12:25:45.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:25:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:25:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:25:46.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:45 smithi090 ceph-mon[33646]: osdmap e725: 8 total, 8 up, 8 in 2024-03-24T12:25:46.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:45 smithi090 ceph-mon[33646]: pgmap v1128: 236 pgs: 236 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:25:46.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:45 smithi090 ceph-mon[33646]: osdmap e726: 8 total, 8 up, 8 in 2024-03-24T12:25:46.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:25:46.209 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:45 smithi006 ceph-mon[30108]: osdmap e725: 8 total, 8 up, 8 in 2024-03-24T12:25:46.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:45 smithi006 ceph-mon[30108]: pgmap v1128: 236 pgs: 236 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:25:46.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:45 smithi006 ceph-mon[30108]: osdmap e726: 8 total, 8 up, 8 in 2024-03-24T12:25:46.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:25:46.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:45 smithi006 ceph-mon[41734]: osdmap e725: 8 total, 8 up, 8 in 2024-03-24T12:25:46.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:45 smithi006 ceph-mon[41734]: pgmap v1128: 236 pgs: 236 active+clean; 4.4 MiB data, 808 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:25:46.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:45 smithi006 ceph-mon[41734]: osdmap e726: 8 total, 8 up, 8 in 2024-03-24T12:25:46.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:25:47.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:25:47.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:47 smithi090 ceph-mon[33646]: osdmap e727: 8 total, 8 up, 8 in 2024-03-24T12:25:47.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:47 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:25:47.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:25:47.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:47 smithi006 ceph-mon[30108]: osdmap e727: 8 total, 8 up, 8 in 2024-03-24T12:25:47.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:47 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:25:47.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:25:47.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:47 smithi006 ceph-mon[41734]: osdmap e727: 8 total, 8 up, 8 in 2024-03-24T12:25:47.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:47 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:25:48.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:48 smithi090 ceph-mon[33646]: pgmap v1131: 268 pgs: 268 active+clean; 4.4 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:25:48.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-146", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:25:48.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:48 smithi090 ceph-mon[33646]: osdmap e728: 8 total, 8 up, 8 in 2024-03-24T12:25:48.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:48 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-146"}]: dispatch 2024-03-24T12:25:48.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:48 smithi006 ceph-mon[30108]: pgmap v1131: 268 pgs: 268 active+clean; 4.4 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:25:48.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-146", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:25:48.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:48 smithi006 ceph-mon[30108]: osdmap e728: 8 total, 8 up, 8 in 2024-03-24T12:25:48.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:48 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-146"}]: dispatch 2024-03-24T12:25:48.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:48 smithi006 ceph-mon[41734]: pgmap v1131: 268 pgs: 268 active+clean; 4.4 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:25:48.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-146", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:25:48.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:48 smithi006 ceph-mon[41734]: osdmap e728: 8 total, 8 up, 8 in 2024-03-24T12:25:48.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:48 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-146"}]: dispatch 2024-03-24T12:25:49.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:49 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-146"}]': finished 2024-03-24T12:25:49.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:49 smithi090 ceph-mon[33646]: osdmap e729: 8 total, 8 up, 8 in 2024-03-24T12:25:49.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:49 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-146", "mode": "writeback"}]: dispatch 2024-03-24T12:25:49.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:49 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:25:49.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:49 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:25:49.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:49 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:25:49.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:49 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-146", "mode": "writeback"}]': finished 2024-03-24T12:25:49.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:49 smithi090 ceph-mon[33646]: osdmap e730: 8 total, 8 up, 8 in 2024-03-24T12:25:49.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:49 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-146"}]': finished 2024-03-24T12:25:49.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:49 smithi006 ceph-mon[41734]: osdmap e729: 8 total, 8 up, 8 in 2024-03-24T12:25:49.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:49 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-146", "mode": "writeback"}]: dispatch 2024-03-24T12:25:49.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:49 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:25:49.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:49 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:25:49.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:49 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:25:49.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:49 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-146", "mode": "writeback"}]': finished 2024-03-24T12:25:49.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:49 smithi006 ceph-mon[41734]: osdmap e730: 8 total, 8 up, 8 in 2024-03-24T12:25:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:49 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi006-82061-111", "overlaypool": "test-rados-api-smithi006-82061-146"}]': finished 2024-03-24T12:25:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:49 smithi006 ceph-mon[30108]: osdmap e729: 8 total, 8 up, 8 in 2024-03-24T12:25:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:49 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-146", "mode": "writeback"}]: dispatch 2024-03-24T12:25:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:49 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:25:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:49 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:25:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:49 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-24T12:25:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:49 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi006-82061-146", "mode": "writeback"}]': finished 2024-03-24T12:25:49.961 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:49 smithi006 ceph-mon[30108]: osdmap e730: 8 total, 8 up, 8 in 2024-03-24T12:25:50.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:50 smithi090 ceph-mon[33646]: pgmap v1134: 268 pgs: 268 active+clean; 4.4 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:25:50.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:50 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:25:50.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:50 smithi006 ceph-mon[41734]: pgmap v1134: 268 pgs: 268 active+clean; 4.4 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:25:50.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:50 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:25:50.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:50 smithi006 ceph-mon[30108]: pgmap v1134: 268 pgs: 268 active+clean; 4.4 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-24T12:25:50.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:50 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "hit_set_count","val": "2"}]: dispatch 2024-03-24T12:25:51.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:51 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:25:51.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:51 smithi090 ceph-mon[33646]: osdmap e731: 8 total, 8 up, 8 in 2024-03-24T12:25:51.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:51 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:25:51.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:51 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:25:51.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:51 smithi006 ceph-mon[41734]: osdmap e731: 8 total, 8 up, 8 in 2024-03-24T12:25:51.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:51 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:25:51.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "hit_set_count","val": "2"}]': finished 2024-03-24T12:25:51.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:51 smithi006 ceph-mon[30108]: osdmap e731: 8 total, 8 up, 8 in 2024-03-24T12:25:51.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:51 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "hit_set_period","val": "600"}]: dispatch 2024-03-24T12:25:52.873 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:52 smithi090 ceph-mon[33646]: pgmap v1137: 268 pgs: 268 active+clean; 4.4 MiB data, 813 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:25:52.873 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:52 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:25:52.873 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:52 smithi090 ceph-mon[33646]: osdmap e732: 8 total, 8 up, 8 in 2024-03-24T12:25:52.873 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:52 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:25:52.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:52 smithi006 ceph-mon[30108]: pgmap v1137: 268 pgs: 268 active+clean; 4.4 MiB data, 813 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:25:52.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:52 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:25:52.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:52 smithi006 ceph-mon[30108]: osdmap e732: 8 total, 8 up, 8 in 2024-03-24T12:25:52.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:52 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:25:52.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:52 smithi006 ceph-mon[41734]: pgmap v1137: 268 pgs: 268 active+clean; 4.4 MiB data, 813 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:25:52.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:52 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "hit_set_period","val": "600"}]': finished 2024-03-24T12:25:52.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:52 smithi006 ceph-mon[41734]: osdmap e732: 8 total, 8 up, 8 in 2024-03-24T12:25:52.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:52 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-24T12:25:53.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:25:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:25:53.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:53 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:25:53.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:53 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "hit_set_type","val": "bloom"}]': finished 2024-03-24T12:25:53.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:53 smithi090 ceph-mon[33646]: osdmap e733: 8 total, 8 up, 8 in 2024-03-24T12:25:53.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:53 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-24T12:25:53.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:53 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-24T12:25:53.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:53 smithi090 ceph-mon[33646]: osdmap e734: 8 total, 8 up, 8 in 2024-03-24T12:25:53.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:53 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "target_max_objects","val": "1"}]: dispatch 2024-03-24T12:25:53.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:53 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:25:53.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:53 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "hit_set_type","val": "bloom"}]': finished 2024-03-24T12:25:53.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:53 smithi006 ceph-mon[30108]: osdmap e733: 8 total, 8 up, 8 in 2024-03-24T12:25:53.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:53 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-24T12:25:53.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:53 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-24T12:25:53.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:53 smithi006 ceph-mon[30108]: osdmap e734: 8 total, 8 up, 8 in 2024-03-24T12:25:53.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:53 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "target_max_objects","val": "1"}]: dispatch 2024-03-24T12:25:53.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:53 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-24T12:25:53.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:53 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "hit_set_type","val": "bloom"}]': finished 2024-03-24T12:25:53.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:53 smithi006 ceph-mon[41734]: osdmap e733: 8 total, 8 up, 8 in 2024-03-24T12:25:53.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:53 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-24T12:25:53.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:53 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-24T12:25:53.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:53 smithi006 ceph-mon[41734]: osdmap e734: 8 total, 8 up, 8 in 2024-03-24T12:25:53.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:53 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "target_max_objects","val": "1"}]: dispatch 2024-03-24T12:25:54.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:54 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:25:54.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:54 smithi090 ceph-mon[33646]: pgmap v1140: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T12:25:54.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "target_max_objects","val": "1"}]': finished 2024-03-24T12:25:54.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:54 smithi090 ceph-mon[33646]: osdmap e735: 8 total, 8 up, 8 in 2024-03-24T12:25:54.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:54 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:25:54.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:54 smithi006 ceph-mon[30108]: pgmap v1140: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T12:25:54.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "target_max_objects","val": "1"}]': finished 2024-03-24T12:25:54.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:54 smithi006 ceph-mon[30108]: osdmap e735: 8 total, 8 up, 8 in 2024-03-24T12:25:54.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:54 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:25:54.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:54 smithi006 ceph-mon[41734]: pgmap v1140: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T12:25:54.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-146","var": "target_max_objects","val": "1"}]': finished 2024-03-24T12:25:54.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:54 smithi006 ceph-mon[41734]: osdmap e735: 8 total, 8 up, 8 in 2024-03-24T12:25:55.625 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:25:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:25:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:25:55.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:55 smithi090 ceph-mon[33646]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-24T12:25:55.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:55 smithi006 ceph-mon[30108]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-24T12:25:55.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:55 smithi006 ceph-mon[41734]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-24T12:25:56.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:56 smithi090 ceph-mon[33646]: pgmap v1143: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T12:25:56.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:56 smithi006 ceph-mon[30108]: pgmap v1143: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T12:25:56.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:56 smithi006 ceph-mon[41734]: pgmap v1143: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T12:25:58.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:25:58 smithi090 ceph-mon[33646]: pgmap v1144: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 2.4 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T12:25:58.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:25:58 smithi006 ceph-mon[30108]: pgmap v1144: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 2.4 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T12:25:58.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:25:58 smithi006 ceph-mon[41734]: pgmap v1144: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 2.4 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-24T12:26:00.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:00 smithi090 ceph-mon[33646]: pgmap v1145: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 757 B/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:26:00.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:00 smithi006 ceph-mon[41734]: pgmap v1145: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 757 B/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:26:00.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:00 smithi006 ceph-mon[30108]: pgmap v1145: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 757 B/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:26:02.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:02 smithi090 ceph-mon[33646]: pgmap v1146: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-03-24T12:26:02.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:26:02 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:26:02.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:02 smithi006 ceph-mon[30108]: pgmap v1146: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-03-24T12:26:02.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:02 smithi006 ceph-mon[41734]: pgmap v1146: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-03-24T12:26:05.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:04 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:26:05.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:04 smithi090 ceph-mon[33646]: pgmap v1147: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:26:05.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:04 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:26:05.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:26:05.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:04 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:05.209 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:04 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:26:05.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:04 smithi006 ceph-mon[30108]: pgmap v1147: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:26:05.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:04 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:26:05.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:26:05.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:04 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:05.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:04 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:26:05.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:04 smithi006 ceph-mon[41734]: pgmap v1147: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:26:05.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:04 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:26:05.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:26:05.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:04 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:05.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:26:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:26:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:26:06.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:05 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:26:06.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:05 smithi090 ceph-mon[33646]: osdmap e736: 8 total, 8 up, 8 in 2024-03-24T12:26:06.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:05 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-146"}]: dispatch 2024-03-24T12:26:06.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:05 smithi090 ceph-mon[33646]: pgmap v1149: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:26:06.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:05 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-146"}]': finished 2024-03-24T12:26:06.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:05 smithi090 ceph-mon[33646]: osdmap e737: 8 total, 8 up, 8 in 2024-03-24T12:26:06.209 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:05 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:26:06.209 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:05 smithi006 ceph-mon[30108]: osdmap e736: 8 total, 8 up, 8 in 2024-03-24T12:26:06.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:05 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-146"}]: dispatch 2024-03-24T12:26:06.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:05 smithi006 ceph-mon[30108]: pgmap v1149: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:26:06.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:05 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-146"}]': finished 2024-03-24T12:26:06.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:05 smithi006 ceph-mon[30108]: osdmap e737: 8 total, 8 up, 8 in 2024-03-24T12:26:06.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:05 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:26:06.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:05 smithi006 ceph-mon[41734]: osdmap e736: 8 total, 8 up, 8 in 2024-03-24T12:26:06.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:05 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-146"}]: dispatch 2024-03-24T12:26:06.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:05 smithi006 ceph-mon[41734]: pgmap v1149: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-24T12:26:06.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:05 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-146"}]': finished 2024-03-24T12:26:06.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:05 smithi006 ceph-mon[41734]: osdmap e737: 8 total, 8 up, 8 in 2024-03-24T12:26:07.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:07 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:07.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:07 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-146"}]: dispatch 2024-03-24T12:26:07.459 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:07 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:07.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:07 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-146"}]: dispatch 2024-03-24T12:26:07.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:07 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:07.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:07 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-146"}]: dispatch 2024-03-24T12:26:08.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:08 smithi090 ceph-mon[33646]: osdmap e738: 8 total, 8 up, 8 in 2024-03-24T12:26:08.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:08 smithi090 ceph-mon[33646]: pgmap v1152: 236 pgs: 236 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:26:08.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:08 smithi090 ceph-mon[33646]: osdmap e739: 8 total, 8 up, 8 in 2024-03-24T12:26:08.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:08 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:26:08.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:08 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:26:08.459 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:08 smithi006 ceph-mon[30108]: osdmap e738: 8 total, 8 up, 8 in 2024-03-24T12:26:08.459 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:08 smithi006 ceph-mon[30108]: pgmap v1152: 236 pgs: 236 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:26:08.459 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:08 smithi006 ceph-mon[30108]: osdmap e739: 8 total, 8 up, 8 in 2024-03-24T12:26:08.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:08 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:26:08.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:08 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:26:08.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:08 smithi006 ceph-mon[41734]: osdmap e738: 8 total, 8 up, 8 in 2024-03-24T12:26:08.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:08 smithi006 ceph-mon[41734]: pgmap v1152: 236 pgs: 236 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:26:08.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:08 smithi006 ceph-mon[41734]: osdmap e739: 8 total, 8 up, 8 in 2024-03-24T12:26:08.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:08 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:26:08.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:08 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:26:09.427 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:26:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=sqlstore.transactions t=2024-03-24T12:26:09.170877121Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-03-24T12:26:09.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:09 smithi090 ceph-mon[33646]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-24T12:26:09.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:09 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:26:09.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:09 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:26:09.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:09 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:26:09.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:09 smithi090 ceph-mon[33646]: osdmap e740: 8 total, 8 up, 8 in 2024-03-24T12:26:09.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:09 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:26:09.459 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:09 smithi006 ceph-mon[30108]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-24T12:26:09.459 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:09 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:26:09.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:09 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:26:09.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:09 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:26:09.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:09 smithi006 ceph-mon[30108]: osdmap e740: 8 total, 8 up, 8 in 2024-03-24T12:26:09.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:09 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:26:09.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:09 smithi006 ceph-mon[41734]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-24T12:26:09.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:09 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:26:09.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:09 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:26:09.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:09 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:26:09.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:09 smithi006 ceph-mon[41734]: osdmap e740: 8 total, 8 up, 8 in 2024-03-24T12:26:09.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:09 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-24T12:26:10.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:10 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:26:10.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:10 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:26:10.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:10 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:26:10.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:10 smithi090 ceph-mon[33646]: pgmap v1155: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-24T12:26:10.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-148", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:26:10.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:10 smithi090 ceph-mon[33646]: osdmap e741: 8 total, 8 up, 8 in 2024-03-24T12:26:10.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:10 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-148"}]: dispatch 2024-03-24T12:26:10.459 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:10 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:26:10.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:10 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:26:10.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:10 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:26:10.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:10 smithi006 ceph-mon[30108]: pgmap v1155: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-24T12:26:10.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-148", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:26:10.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:10 smithi006 ceph-mon[30108]: osdmap e741: 8 total, 8 up, 8 in 2024-03-24T12:26:10.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:10 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-148"}]: dispatch 2024-03-24T12:26:10.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:10 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:26:10.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:10 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:26:10.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:10 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:26:10.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:10 smithi006 ceph-mon[41734]: pgmap v1155: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-24T12:26:10.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-148", "force_nonempty": "--force-nonempty" }]': finished 2024-03-24T12:26:10.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:10 smithi006 ceph-mon[41734]: osdmap e741: 8 total, 8 up, 8 in 2024-03-24T12:26:10.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:10 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-148"}]: dispatch 2024-03-24T12:26:12.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-148"}]': finished 2024-03-24T12:26:12.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:11 smithi090 ceph-mon[33646]: osdmap e742: 8 total, 8 up, 8 in 2024-03-24T12:26:12.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:12.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:11 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-148"}]: dispatch 2024-03-24T12:26:12.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:11 smithi090 ceph-mon[33646]: pgmap v1158: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:26:12.209 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-148"}]': finished 2024-03-24T12:26:12.209 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:11 smithi006 ceph-mon[30108]: osdmap e742: 8 total, 8 up, 8 in 2024-03-24T12:26:12.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:12.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:11 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-148"}]: dispatch 2024-03-24T12:26:12.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:11 smithi006 ceph-mon[30108]: pgmap v1158: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:26:12.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-148"}]': finished 2024-03-24T12:26:12.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:11 smithi006 ceph-mon[41734]: osdmap e742: 8 total, 8 up, 8 in 2024-03-24T12:26:12.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:12.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:11 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-148"}]: dispatch 2024-03-24T12:26:12.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:11 smithi006 ceph-mon[41734]: pgmap v1158: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:26:13.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:12 smithi090 ceph-mon[33646]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:26:13.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:12 smithi090 ceph-mon[33646]: osdmap e743: 8 total, 8 up, 8 in 2024-03-24T12:26:13.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:26:12 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:26:13.209 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:12 smithi006 ceph-mon[30108]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:26:13.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:12 smithi006 ceph-mon[30108]: osdmap e743: 8 total, 8 up, 8 in 2024-03-24T12:26:13.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:12 smithi006 ceph-mon[41734]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:26:13.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:12 smithi006 ceph-mon[41734]: osdmap e743: 8 total, 8 up, 8 in 2024-03-24T12:26:14.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:13 smithi090 ceph-mon[33646]: osdmap e744: 8 total, 8 up, 8 in 2024-03-24T12:26:14.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:13 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:26:14.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:13 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:26:14.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:13 smithi090 ceph-mon[33646]: pgmap v1161: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:26:14.209 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:13 smithi006 ceph-mon[30108]: osdmap e744: 8 total, 8 up, 8 in 2024-03-24T12:26:14.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:13 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:26:14.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:13 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:26:14.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:13 smithi006 ceph-mon[30108]: pgmap v1161: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:26:14.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:13 smithi006 ceph-mon[41734]: osdmap e744: 8 total, 8 up, 8 in 2024-03-24T12:26:14.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:13 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:26:14.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:13 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:26:14.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:13 smithi006 ceph-mon[41734]: pgmap v1161: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:26:15.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:26:15.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:15 smithi090 ceph-mon[33646]: osdmap e745: 8 total, 8 up, 8 in 2024-03-24T12:26:15.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:15.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:15 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-150"}]: dispatch 2024-03-24T12:26:15.459 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:26:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:26:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:26:15.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:26:15.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:15 smithi006 ceph-mon[30108]: osdmap e745: 8 total, 8 up, 8 in 2024-03-24T12:26:15.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:15.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:15 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-150"}]: dispatch 2024-03-24T12:26:15.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:26:15.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:15 smithi006 ceph-mon[41734]: osdmap e745: 8 total, 8 up, 8 in 2024-03-24T12:26:15.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:15.461 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:15 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-150"}]: dispatch 2024-03-24T12:26:16.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:16 smithi090 ceph-mon[33646]: osdmap e746: 8 total, 8 up, 8 in 2024-03-24T12:26:16.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:16 smithi090 ceph-mon[33646]: pgmap v1164: 236 pgs: 236 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:26:16.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:16 smithi090 ceph-mon[33646]: osdmap e747: 8 total, 8 up, 8 in 2024-03-24T12:26:16.427 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:16 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:26:16.459 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:16 smithi006 ceph-mon[30108]: osdmap e746: 8 total, 8 up, 8 in 2024-03-24T12:26:16.459 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:16 smithi006 ceph-mon[30108]: pgmap v1164: 236 pgs: 236 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:26:16.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:16 smithi006 ceph-mon[30108]: osdmap e747: 8 total, 8 up, 8 in 2024-03-24T12:26:16.460 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:16 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:26:16.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:16 smithi006 ceph-mon[41734]: osdmap e746: 8 total, 8 up, 8 in 2024-03-24T12:26:16.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:16 smithi006 ceph-mon[41734]: pgmap v1164: 236 pgs: 236 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-24T12:26:16.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:16 smithi006 ceph-mon[41734]: osdmap e747: 8 total, 8 up, 8 in 2024-03-24T12:26:16.460 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:16 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:26:18.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:26:18.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:17 smithi090 ceph-mon[33646]: osdmap e748: 8 total, 8 up, 8 in 2024-03-24T12:26:18.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:18.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-152"}]: dispatch 2024-03-24T12:26:18.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:17 smithi090 ceph-mon[33646]: pgmap v1167: 268 pgs: 268 active+clean; 4.4 MiB data, 851 MiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-24T12:26:18.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:26:18.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:17 smithi006 ceph-mon[30108]: osdmap e748: 8 total, 8 up, 8 in 2024-03-24T12:26:18.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:18.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-152"}]: dispatch 2024-03-24T12:26:18.212 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:17 smithi006 ceph-mon[30108]: pgmap v1167: 268 pgs: 268 active+clean; 4.4 MiB data, 851 MiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-24T12:26:18.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:26:18.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:17 smithi006 ceph-mon[41734]: osdmap e748: 8 total, 8 up, 8 in 2024-03-24T12:26:18.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:18.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-152"}]: dispatch 2024-03-24T12:26:18.213 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:17 smithi006 ceph-mon[41734]: pgmap v1167: 268 pgs: 268 active+clean; 4.4 MiB data, 851 MiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-24T12:26:19.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:18 smithi090 ceph-mon[33646]: osdmap e749: 8 total, 8 up, 8 in 2024-03-24T12:26:19.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:18 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:26:19.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:18 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:26:19.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:18 smithi006 ceph-mon[30108]: osdmap e749: 8 total, 8 up, 8 in 2024-03-24T12:26:19.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:18 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:26:19.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:18 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:26:19.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:18 smithi006 ceph-mon[41734]: osdmap e749: 8 total, 8 up, 8 in 2024-03-24T12:26:19.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:18 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:26:19.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:18 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:26:20.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:19 smithi090 ceph-mon[33646]: osdmap e750: 8 total, 8 up, 8 in 2024-03-24T12:26:20.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:26:20.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:19 smithi090 ceph-mon[33646]: pgmap v1170: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 851 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-03-24T12:26:20.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:26:20.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:19 smithi090 ceph-mon[33646]: osdmap e751: 8 total, 8 up, 8 in 2024-03-24T12:26:20.209 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:19 smithi006 ceph-mon[30108]: osdmap e750: 8 total, 8 up, 8 in 2024-03-24T12:26:20.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:26:20.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:19 smithi006 ceph-mon[30108]: pgmap v1170: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 851 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-03-24T12:26:20.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:26:20.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:19 smithi006 ceph-mon[30108]: osdmap e751: 8 total, 8 up, 8 in 2024-03-24T12:26:20.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:19 smithi006 ceph-mon[41734]: osdmap e750: 8 total, 8 up, 8 in 2024-03-24T12:26:20.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:26:20.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:19 smithi006 ceph-mon[41734]: pgmap v1170: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 851 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-03-24T12:26:20.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi006-82061-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:26:20.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:19 smithi006 ceph-mon[41734]: osdmap e751: 8 total, 8 up, 8 in 2024-03-24T12:26:21.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:20 smithi090 ceph-mon[33646]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:26:21.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:20 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-111","var": "dedup_tier","val": "test-rados-api-smithi006-82061-154"}]: dispatch 2024-03-24T12:26:21.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:20 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:21.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:20 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-154"}]: dispatch 2024-03-24T12:26:21.209 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:20 smithi006 ceph-mon[30108]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:26:21.209 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:20 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-111","var": "dedup_tier","val": "test-rados-api-smithi006-82061-154"}]: dispatch 2024-03-24T12:26:21.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:20 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:21.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:20 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-154"}]: dispatch 2024-03-24T12:26:21.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:20 smithi006 ceph-mon[41734]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:26:21.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:20 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi006-82061-111","var": "dedup_tier","val": "test-rados-api-smithi006-82061-154"}]: dispatch 2024-03-24T12:26:21.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:20 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:21.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:20 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi006-82061-111", "tierpool": "test-rados-api-smithi006-82061-154"}]: dispatch 2024-03-24T12:26:21.591 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.591 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.591 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.591 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.591 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.591 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.591 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.591 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.591 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.591 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.592 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.592 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.592 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.592 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.592 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.592 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.592 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.592 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.592 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.592 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.592 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.592 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.592 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.592 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.593 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.593 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.593 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.593 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.593 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.593 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.593 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.593 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.593 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.593 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.593 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.593 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.593 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.593 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.594 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.594 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.594 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.594 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.594 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.594 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.594 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.594 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.594 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.594 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.594 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.594 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.594 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.594 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.594 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.595 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.595 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.595 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.595 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.595 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.595 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.595 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.595 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.595 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.595 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.595 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.595 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.595 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.595 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.595 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.596 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.596 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.596 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.596 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.596 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.596 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.596 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.596 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.596 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.596 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.596 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.596 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.596 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.596 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.596 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.597 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.597 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.597 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.597 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.597 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.597 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.597 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.597 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.597 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.597 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.597 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.597 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.597 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.597 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.597 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.597 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:21.598 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:26:22.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:21 smithi090 ceph-mon[33646]: osdmap e752: 8 total, 8 up, 8 in 2024-03-24T12:26:22.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:21 smithi090 ceph-mon[33646]: pgmap v1173: 236 pgs: 236 active+clean; 4.4 MiB data, 851 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:26:22.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:21 smithi090 ceph-mon[33646]: osdmap e753: 8 total, 8 up, 8 in 2024-03-24T12:26:22.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:22.209 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:21 smithi006 ceph-mon[30108]: osdmap e752: 8 total, 8 up, 8 in 2024-03-24T12:26:22.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:21 smithi006 ceph-mon[30108]: pgmap v1173: 236 pgs: 236 active+clean; 4.4 MiB data, 851 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:26:22.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:21 smithi006 ceph-mon[30108]: osdmap e753: 8 total, 8 up, 8 in 2024-03-24T12:26:22.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:22.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:21 smithi006 ceph-mon[41734]: osdmap e752: 8 total, 8 up, 8 in 2024-03-24T12:26:22.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:21 smithi006 ceph-mon[41734]: pgmap v1173: 236 pgs: 236 active+clean; 4.4 MiB data, 851 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:26:22.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:21 smithi006 ceph-mon[41734]: osdmap e753: 8 total, 8 up, 8 in 2024-03-24T12:26:22.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:23.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:26:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:26:23.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waitLibRadosTwoPoolsECPP.TryFlush (7831 ms) 2024-03-24T12:26:23.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-03-24T12:26:23.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (12389 ms) 2024-03-24T12:26:23.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-03-24T12:26:23.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8052 ms) 2024-03-24T12:26:23.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-03-24T12:26:23.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13086 ms) 2024-03-24T12:26:23.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-03-24T12:26:23.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7482 ms) 2024-03-24T12:26:23.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-03-24T12:26:23.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8062 ms) 2024-03-24T12:26:23.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-03-24T12:26:23.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: hmm, no HitSet yet 2024-03-24T12:26:23.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: ok, hit_set contains 336:602f83fe:::foo:head 2024-03-24T12:26:23.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9062 ms) 2024-03-24T12:26:23.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-03-24T12:26:23.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711283099,0 2024-03-24T12:26:23.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: first is 1711283099 2024-03-24T12:26:23.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711283099,0 2024-03-24T12:26:23.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711283099,0 2024-03-24T12:26:23.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711283099,1711283101,1711283102,0 2024-03-24T12:26:23.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711283099,1711283101,1711283102,0 2024-03-24T12:26:23.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711283099,1711283101,1711283102,0 2024-03-24T12:26:23.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711283099,1711283101,1711283102,1711283104,1711283105,0 2024-03-24T12:26:23.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711283099,1711283101,1711283102,1711283104,1711283105,0 2024-03-24T12:26:23.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711283099,1711283101,1711283102,1711283104,1711283105,0 2024-03-24T12:26:23.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711283099,1711283101,1711283102,1711283104,1711283105,1711283107,1711283108,0 2024-03-24T12:26:23.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711283099,1711283101,1711283102,1711283104,1711283105,1711283107,1711283108,0 2024-03-24T12:26:23.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711283099,1711283101,1711283102,1711283104,1711283105,1711283107,1711283108,0 2024-03-24T12:26:23.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: got ls 1711283102,1711283104,1711283105,1711283107,1711283108,1711283110,1711283111,0 2024-03-24T12:26:23.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: first now 1711283102, trimmed 2024-03-24T12:26:23.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (20299 ms) 2024-03-24T12:26:23.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-03-24T12:26:23.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: foo0 2024-03-24T12:26:23.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-03-24T12:26:23.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (15063 ms) 2024-03-24T12:26:23.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-03-24T12:26:23.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (18055 ms) 2024-03-24T12:26:23.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-03-24T12:26:23.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (22241 ms) 2024-03-24T12:26:23.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-03-24T12:26:23.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5017 ms) 2024-03-24T12:26:23.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-03-24T12:26:23.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3023 ms) 2024-03-24T12:26:23.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-03-24T12:26:23.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3009 ms) 2024-03-24T12:26:23.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-03-24T12:26:23.844 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3024 ms) 2024-03-24T12:26:23.844 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (230931 ms total) 2024-03-24T12:26:23.844 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: 2024-03-24T12:26:23.844 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-03-24T12:26:23.844 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (862346 ms total) 2024-03-24T12:26:23.844 INFO:tasks.workunit.client.0.smithi006.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-03-24T12:26:23.845 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:26:23.845 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81749 2024-03-24T12:26:23.845 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 81749 2024-03-24T12:26:23.845 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:26:23.845 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81901 2024-03-24T12:26:23.845 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 81901 2024-03-24T12:26:23.846 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:26:23.846 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82480 2024-03-24T12:26:23.846 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82480 2024-03-24T12:26:23.846 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:26:23.846 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82163 2024-03-24T12:26:23.846 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82163 2024-03-24T12:26:23.846 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:26:23.846 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82031 2024-03-24T12:26:23.846 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82031 2024-03-24T12:26:23.846 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:26:23.846 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81836 2024-03-24T12:26:23.846 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 81836 2024-03-24T12:26:24.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:26:24.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:23 smithi090 ceph-mon[33646]: osdmap e754: 8 total, 8 up, 8 in 2024-03-24T12:26:24.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:24.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:23 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:26:24.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:23 smithi090 ceph-mon[33646]: pgmap v1176: 228 pgs: 228 active+clean; 582 KiB data, 869 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:26:24.209 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:26:24.209 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:23 smithi006 ceph-mon[30108]: osdmap e754: 8 total, 8 up, 8 in 2024-03-24T12:26:24.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:24.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:23 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:26:24.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:23 smithi006 ceph-mon[30108]: pgmap v1176: 228 pgs: 228 active+clean; 582 KiB data, 869 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:26:24.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:26:24.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:23 smithi006 ceph-mon[41734]: osdmap e754: 8 total, 8 up, 8 in 2024-03-24T12:26:24.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-111"}]: dispatch 2024-03-24T12:26:24.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:23 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:26:24.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:23 smithi006 ceph-mon[41734]: pgmap v1176: 228 pgs: 228 active+clean; 582 KiB data, 869 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:26:25.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:24 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:26:25.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:24 smithi090 ceph-mon[33646]: osdmap e755: 8 total, 8 up, 8 in 2024-03-24T12:26:25.209 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:24 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:26:25.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:24 smithi006 ceph-mon[30108]: osdmap e755: 8 total, 8 up, 8 in 2024-03-24T12:26:25.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:24 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2913143830' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi006-82061-111"}]': finished 2024-03-24T12:26:25.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:24 smithi006 ceph-mon[41734]: osdmap e755: 8 total, 8 up, 8 in 2024-03-24T12:26:25.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:26:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:26:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:26:26.176 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:25 smithi090 ceph-mon[33646]: pgmap v1178: 228 pgs: 228 active+clean; 582 KiB data, 869 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:26:26.209 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:25 smithi006 ceph-mon[30108]: pgmap v1178: 228 pgs: 228 active+clean; 582 KiB data, 869 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:26:26.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:25 smithi006 ceph-mon[41734]: pgmap v1178: 228 pgs: 228 active+clean; 582 KiB data, 869 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:26:27.176 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:26 smithi090 ceph-mon[33646]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:26:27.209 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:26 smithi006 ceph-mon[30108]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:26:27.209 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:26 smithi006 ceph-mon[41734]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:26:28.176 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:27 smithi090 ceph-mon[33646]: pgmap v1179: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:26:28.209 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:27 smithi006 ceph-mon[30108]: pgmap v1179: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:26:28.209 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:27 smithi006 ceph-mon[41734]: pgmap v1179: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:26:30.676 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:30 smithi090 ceph-mon[33646]: pgmap v1180: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T12:26:30.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:30 smithi006 ceph-mon[30108]: pgmap v1180: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T12:26:30.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:30 smithi006 ceph-mon[41734]: pgmap v1180: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T12:26:32.676 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:32 smithi090 ceph-mon[33646]: pgmap v1181: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 602 B/s rd, 0 op/s 2024-03-24T12:26:32.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:32 smithi006 ceph-mon[30108]: pgmap v1181: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 602 B/s rd, 0 op/s 2024-03-24T12:26:32.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:32 smithi006 ceph-mon[41734]: pgmap v1181: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 602 B/s rd, 0 op/s 2024-03-24T12:26:33.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:26:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:26:34.676 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:34 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:26:34.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:34 smithi090 ceph-mon[33646]: pgmap v1182: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:26:34.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:26:34.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:34 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:26:34.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:34 smithi006 ceph-mon[30108]: pgmap v1182: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:26:34.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:26:34.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:34 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:26:34.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:34 smithi006 ceph-mon[41734]: pgmap v1182: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:26:34.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:26:35.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:26:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:26:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:26:36.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:36 smithi090 ceph-mon[33646]: pgmap v1183: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 890 B/s rd, 0 op/s 2024-03-24T12:26:36.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:36 smithi006 ceph-mon[30108]: pgmap v1183: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 890 B/s rd, 0 op/s 2024-03-24T12:26:36.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:36 smithi006 ceph-mon[41734]: pgmap v1183: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 890 B/s rd, 0 op/s 2024-03-24T12:26:38.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:38 smithi090 ceph-mon[33646]: pgmap v1184: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:26:38.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:38 smithi006 ceph-mon[30108]: pgmap v1184: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:26:38.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:38 smithi006 ceph-mon[41734]: pgmap v1184: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:26:40.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:40 smithi090 ceph-mon[33646]: pgmap v1185: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:26:40.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:40 smithi006 ceph-mon[30108]: pgmap v1185: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:26:40.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:40 smithi006 ceph-mon[41734]: pgmap v1185: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:26:42.676 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:42 smithi090 ceph-mon[33646]: pgmap v1186: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:26:42.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:42 smithi006 ceph-mon[30108]: pgmap v1186: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:26:42.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:42 smithi006 ceph-mon[41734]: pgmap v1186: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:26:43.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:26:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:26:44.676 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:44 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:26:44.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:44 smithi090 ceph-mon[33646]: pgmap v1187: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:26:44.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:44 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:26:44.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:44 smithi006 ceph-mon[30108]: pgmap v1187: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:26:44.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:44 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:26:44.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:44 smithi006 ceph-mon[41734]: pgmap v1187: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:26:45.710 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:26:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:26:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:26:46.676 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:46 smithi090 ceph-mon[33646]: pgmap v1188: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:26:46.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:46 smithi006 ceph-mon[30108]: pgmap v1188: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:26:46.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:46 smithi006 ceph-mon[41734]: pgmap v1188: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:26:48.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:48 smithi090 ceph-mon[33646]: pgmap v1189: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:26:48.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:48 smithi006 ceph-mon[30108]: pgmap v1189: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:26:48.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:48 smithi006 ceph-mon[41734]: pgmap v1189: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:26:49.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:49 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:26:49.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:49 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:26:49.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:49 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:26:50.676 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:50 smithi090 ceph-mon[33646]: pgmap v1190: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:26:50.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:50 smithi006 ceph-mon[30108]: pgmap v1190: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:26:50.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:50 smithi006 ceph-mon[41734]: pgmap v1190: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:26:52.676 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:52 smithi090 ceph-mon[33646]: pgmap v1191: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:26:52.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:52 smithi006 ceph-mon[30108]: pgmap v1191: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:26:52.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:52 smithi006 ceph-mon[41734]: pgmap v1191: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:26:53.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:26:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:26:54.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:54 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:26:54.677 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:54 smithi090 ceph-mon[33646]: pgmap v1192: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:26:54.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:54 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:26:54.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:54 smithi006 ceph-mon[30108]: pgmap v1192: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:26:54.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:54 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:26:54.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:54 smithi006 ceph-mon[41734]: pgmap v1192: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:26:55.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:26:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:26:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:26:56.676 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:56 smithi090 ceph-mon[33646]: pgmap v1193: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:26:56.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:56 smithi006 ceph-mon[30108]: pgmap v1193: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:26:56.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:56 smithi006 ceph-mon[41734]: pgmap v1193: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:26:58.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:26:58 smithi006 ceph-mon[30108]: pgmap v1194: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:26:58.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:26:58 smithi006 ceph-mon[41734]: pgmap v1194: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:26:58.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:26:58 smithi090 ceph-mon[33646]: pgmap v1194: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:00.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:00 smithi006 ceph-mon[30108]: pgmap v1195: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:27:00.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:00 smithi006 ceph-mon[41734]: pgmap v1195: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:27:00.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:00 smithi090 ceph-mon[33646]: pgmap v1195: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:27:02.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:02 smithi006 ceph-mon[30108]: pgmap v1196: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:02.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:02 smithi006 ceph-mon[41734]: pgmap v1196: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:02.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:02 smithi090 ceph-mon[33646]: pgmap v1196: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:03.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:27:02 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:27:04.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:04 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:27:04.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:04 smithi006 ceph-mon[30108]: pgmap v1197: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:04.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:27:04.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:04 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:27:04.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:04 smithi006 ceph-mon[41734]: pgmap v1197: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:04.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:27:04.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:04 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:27:04.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:04 smithi090 ceph-mon[33646]: pgmap v1197: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:04.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:27:05.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:27:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:27:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:27:06.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:06 smithi006 ceph-mon[30108]: pgmap v1198: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:06.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:06 smithi006 ceph-mon[41734]: pgmap v1198: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:06.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:06 smithi090 ceph-mon[33646]: pgmap v1198: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:08.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:08 smithi006 ceph-mon[30108]: pgmap v1199: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:08.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:08 smithi006 ceph-mon[41734]: pgmap v1199: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:08.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:08 smithi090 ceph-mon[33646]: pgmap v1199: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:09.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:09 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:27:09.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:09 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:27:09.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:09 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:27:10.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:10 smithi006 ceph-mon[30108]: pgmap v1200: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:10.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:10 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:27:10.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:10 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:27:10.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:10 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:27:10.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:10 smithi006 ceph-mon[41734]: pgmap v1200: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:10.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:10 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:27:10.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:10 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:27:10.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:10 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:27:10.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:10 smithi090 ceph-mon[33646]: pgmap v1200: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:10.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:10 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:27:10.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:10 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:27:10.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:10 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:27:12.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:12 smithi006 ceph-mon[41734]: pgmap v1201: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:12.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:12 smithi006 ceph-mon[30108]: pgmap v1201: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:12.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:12 smithi090 ceph-mon[33646]: pgmap v1201: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:13.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:27:12 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:27:14.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:14 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:27:14.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:14 smithi006 ceph-mon[30108]: pgmap v1202: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:14.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:14 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:27:14.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:14 smithi006 ceph-mon[41734]: pgmap v1202: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:14.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:14 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:27:14.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:14 smithi090 ceph-mon[33646]: pgmap v1202: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:15.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:27:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:27:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:27:16.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:16 smithi006 ceph-mon[30108]: pgmap v1203: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:16.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:16 smithi006 ceph-mon[41734]: pgmap v1203: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:16.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:16 smithi090 ceph-mon[33646]: pgmap v1203: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:18.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:18 smithi006 ceph-mon[30108]: pgmap v1204: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:18.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:18 smithi006 ceph-mon[41734]: pgmap v1204: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:18.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:18 smithi090 ceph-mon[33646]: pgmap v1204: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:19.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:27:19.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:27:19.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:27:20.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:20 smithi090 ceph-mon[33646]: pgmap v1205: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:20.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:20 smithi006 ceph-mon[30108]: pgmap v1205: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:20.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:20 smithi006 ceph-mon[41734]: pgmap v1205: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:22.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:22 smithi090 ceph-mon[33646]: pgmap v1206: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:22.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:22 smithi006 ceph-mon[30108]: pgmap v1206: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:22.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:22 smithi006 ceph-mon[41734]: pgmap v1206: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:23.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:27:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:27:24.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:24 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:27:24.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:24 smithi090 ceph-mon[33646]: pgmap v1207: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:24.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:24 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:27:24.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:24 smithi006 ceph-mon[30108]: pgmap v1207: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:24.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:24 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:27:24.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:24 smithi006 ceph-mon[41734]: pgmap v1207: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:25.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:27:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:27:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:27:26.729 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:26 smithi090 ceph-mon[33646]: pgmap v1208: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:27:26.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:26 smithi006 ceph-mon[30108]: pgmap v1208: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:27:26.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:26 smithi006 ceph-mon[41734]: pgmap v1208: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:27:28.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:28 smithi090 ceph-mon[33646]: pgmap v1209: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:28.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:28 smithi006 ceph-mon[30108]: pgmap v1209: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:28.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:28 smithi006 ceph-mon[41734]: pgmap v1209: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:30.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:30 smithi090 ceph-mon[33646]: pgmap v1210: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:27:30.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:30 smithi006 ceph-mon[30108]: pgmap v1210: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:27:30.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:30 smithi006 ceph-mon[41734]: pgmap v1210: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:27:32.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:32 smithi090 ceph-mon[33646]: pgmap v1211: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:32.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:32 smithi006 ceph-mon[30108]: pgmap v1211: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:32.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:32 smithi006 ceph-mon[41734]: pgmap v1211: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:33.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:27:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:27:34.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:34 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:27:34.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:34 smithi090 ceph-mon[33646]: pgmap v1212: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:34.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:27:34.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:34 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:27:34.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:34 smithi006 ceph-mon[30108]: pgmap v1212: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:34.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:27:34.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:34 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:27:34.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:34 smithi006 ceph-mon[41734]: pgmap v1212: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:34.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:27:35.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:27:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:27:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:27:36.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:36 smithi090 ceph-mon[33646]: pgmap v1213: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:27:36.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:36 smithi006 ceph-mon[30108]: pgmap v1213: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:27:36.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:36 smithi006 ceph-mon[41734]: pgmap v1213: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:27:38.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:38 smithi090 ceph-mon[33646]: pgmap v1214: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:38.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:38 smithi006 ceph-mon[30108]: pgmap v1214: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:38.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:38 smithi006 ceph-mon[41734]: pgmap v1214: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:40.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:40 smithi090 ceph-mon[33646]: pgmap v1215: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:27:40.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:40 smithi006 ceph-mon[30108]: pgmap v1215: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:27:40.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:40 smithi006 ceph-mon[41734]: pgmap v1215: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:27:42.761 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:42 smithi006 ceph-mon[30108]: pgmap v1216: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:42.762 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:42 smithi006 ceph-mon[41734]: pgmap v1216: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:42.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:42 smithi090 ceph-mon[33646]: pgmap v1216: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:43.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:27:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:27:44.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:44 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:27:44.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:44 smithi090 ceph-mon[33646]: pgmap v1217: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:44.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:44 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:27:44.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:44 smithi006 ceph-mon[30108]: pgmap v1217: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:44.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:44 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:27:44.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:44 smithi006 ceph-mon[41734]: pgmap v1217: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:45.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:27:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:27:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:27:46.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:46 smithi090 ceph-mon[33646]: pgmap v1218: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:27:46.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:46 smithi006 ceph-mon[30108]: pgmap v1218: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:27:46.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:46 smithi006 ceph-mon[41734]: pgmap v1218: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:27:48.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:48 smithi090 ceph-mon[33646]: pgmap v1219: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:48.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:48 smithi006 ceph-mon[30108]: pgmap v1219: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:48.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:48 smithi006 ceph-mon[41734]: pgmap v1219: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:49.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:49 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:27:49.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:49 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:27:49.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:49 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:27:50.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:50 smithi090 ceph-mon[33646]: pgmap v1220: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:27:50.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:50 smithi006 ceph-mon[30108]: pgmap v1220: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:27:50.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:50 smithi006 ceph-mon[41734]: pgmap v1220: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:27:52.426 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:52 smithi090 ceph-mon[33646]: pgmap v1221: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:52.459 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:52 smithi006 ceph-mon[30108]: pgmap v1221: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:52.459 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:52 smithi006 ceph-mon[41734]: pgmap v1221: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:53.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:27:52 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:27:54.676 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:54 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:27:54.676 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:54 smithi090 ceph-mon[33646]: pgmap v1222: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:54.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:54 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:27:54.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:54 smithi006 ceph-mon[30108]: pgmap v1222: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:54.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:54 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:27:54.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:54 smithi006 ceph-mon[41734]: pgmap v1222: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:55.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:27:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:27:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:27:56.676 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:56 smithi090 ceph-mon[33646]: pgmap v1223: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:56.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:56 smithi006 ceph-mon[30108]: pgmap v1223: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:56.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:56 smithi006 ceph-mon[41734]: pgmap v1223: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:27:58.676 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:27:58 smithi090 ceph-mon[33646]: pgmap v1224: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:58.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:27:58 smithi006 ceph-mon[30108]: pgmap v1224: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:27:58.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:27:58 smithi006 ceph-mon[41734]: pgmap v1224: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:00.676 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:00 smithi090 ceph-mon[33646]: pgmap v1225: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:28:00.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:00 smithi006 ceph-mon[30108]: pgmap v1225: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:28:00.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:00 smithi006 ceph-mon[41734]: pgmap v1225: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:28:01.603 INFO:tasks.workunit.client.0.smithi006.stdout:ing 1 sec 2024-03-24T12:28:01.603 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.603 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.603 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.604 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.604 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.604 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.604 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.604 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.604 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.604 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.604 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.604 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.604 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.604 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.604 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.605 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.605 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.605 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.605 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.605 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.605 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.605 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.605 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.605 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.605 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.605 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.605 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.605 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.605 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.605 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.606 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.607 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.607 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.607 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.607 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.607 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.607 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.607 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.607 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.607 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.607 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.607 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.607 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.607 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.607 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.608 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.608 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.608 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.608 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.608 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.608 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.608 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.608 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.608 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.608 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.608 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.608 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.608 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.608 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.608 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.608 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.609 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.609 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.609 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.609 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.609 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.609 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.609 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.609 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.609 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.609 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.609 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.609 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.609 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.609 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.610 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.610 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.610 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.610 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.610 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.610 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.610 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.610 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.610 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.610 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:01.610 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:28:02.676 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:02 smithi090 ceph-mon[33646]: pgmap v1226: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:02.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:02 smithi006 ceph-mon[30108]: pgmap v1226: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:02.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:02 smithi006 ceph-mon[41734]: pgmap v1226: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:03.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:28:02 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:28:04.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:04 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:28:04.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:04 smithi006 ceph-mon[41734]: pgmap v1227: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:04.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:28:04.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:04 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:28:04.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:04 smithi006 ceph-mon[30108]: pgmap v1227: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:04.710 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:28:04.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:04 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:28:04.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:04 smithi090 ceph-mon[33646]: pgmap v1227: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:04.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:28:05.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:28:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:28:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:28:06.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:06 smithi006 ceph-mon[30108]: pgmap v1228: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:06.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:06 smithi006 ceph-mon[41734]: pgmap v1228: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:06.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:06 smithi090 ceph-mon[33646]: pgmap v1228: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:08.676 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:08 smithi090 ceph-mon[33646]: pgmap v1229: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:08.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:08 smithi006 ceph-mon[30108]: pgmap v1229: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:08.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:08 smithi006 ceph-mon[41734]: pgmap v1229: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:10.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:10 smithi006 ceph-mon[30108]: pgmap v1230: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:10.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:10 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:28:10.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:10 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:28:10.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:10 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:28:10.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:10 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:28:10.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:10 smithi006 ceph-mon[41734]: pgmap v1230: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:10.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:10 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:28:10.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:10 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:28:10.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:10 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:28:10.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:10 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:28:10.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:10 smithi090 ceph-mon[33646]: pgmap v1230: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:10.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:10 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:28:10.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:10 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:28:10.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:10 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:28:10.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:10 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:28:12.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:12 smithi006 ceph-mon[30108]: pgmap v1231: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:12.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:12 smithi006 ceph-mon[41734]: pgmap v1231: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:12.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:12 smithi090 ceph-mon[33646]: pgmap v1231: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:13.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:28:13 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:28:14.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:14 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:28:14.711 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:14 smithi006 ceph-mon[30108]: pgmap v1232: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:14.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:14 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:28:14.711 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:14 smithi006 ceph-mon[41734]: pgmap v1232: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:14.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:14 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:28:14.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:14 smithi090 ceph-mon[33646]: pgmap v1232: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:15.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:28:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:28:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:28:16.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:16 smithi006 ceph-mon[30108]: pgmap v1233: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:16.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:16 smithi006 ceph-mon[41734]: pgmap v1233: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:16.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:16 smithi090 ceph-mon[33646]: pgmap v1233: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:18.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:18 smithi006 ceph-mon[30108]: pgmap v1234: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:18.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:18 smithi006 ceph-mon[41734]: pgmap v1234: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:18.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:18 smithi090 ceph-mon[33646]: pgmap v1234: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:19.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:28:19.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:28:19.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:28:20.709 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:20 smithi006 ceph-mon[30108]: pgmap v1235: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:20.709 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:20 smithi006 ceph-mon[41734]: pgmap v1235: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:20.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:20 smithi090 ceph-mon[33646]: pgmap v1235: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:22.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:22 smithi090 ceph-mon[33646]: pgmap v1236: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:22.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:22 smithi006 ceph-mon[30108]: pgmap v1236: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:22.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:22 smithi006 ceph-mon[41734]: pgmap v1236: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:23.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:28:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:28:24.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:24 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:28:24.930 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:24 smithi090 ceph-mon[33646]: pgmap v1237: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:24.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:24 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:28:24.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:24 smithi006 ceph-mon[30108]: pgmap v1237: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:24.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:24 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:28:24.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:24 smithi006 ceph-mon[41734]: pgmap v1237: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:25.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:28:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:28:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:28:26.729 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:26 smithi090 ceph-mon[33646]: pgmap v1238: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:26.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:26 smithi006 ceph-mon[30108]: pgmap v1238: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:26.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:26 smithi006 ceph-mon[41734]: pgmap v1238: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:28.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:28 smithi090 ceph-mon[33646]: pgmap v1239: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:28.960 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:28 smithi006 ceph-mon[30108]: pgmap v1239: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:28.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:28 smithi006 ceph-mon[41734]: pgmap v1239: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:30.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:30 smithi090 ceph-mon[33646]: pgmap v1240: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:30.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:30 smithi006 ceph-mon[30108]: pgmap v1240: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:30.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:30 smithi006 ceph-mon[41734]: pgmap v1240: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:32.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:32 smithi090 ceph-mon[33646]: pgmap v1241: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:32.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:32 smithi006 ceph-mon[41734]: pgmap v1241: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:32.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:32 smithi006 ceph-mon[30108]: pgmap v1241: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:33.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:28:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:28:34.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:34 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:28:34.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:34 smithi090 ceph-mon[33646]: pgmap v1242: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:34.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:28:34.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:34 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:28:34.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:34 smithi006 ceph-mon[30108]: pgmap v1242: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:34.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:28:34.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:34 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:28:34.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:34 smithi006 ceph-mon[41734]: pgmap v1242: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:34.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:28:35.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:28:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:28:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:28:36.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:36 smithi090 ceph-mon[33646]: pgmap v1243: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:28:36.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:36 smithi006 ceph-mon[30108]: pgmap v1243: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:28:36.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:36 smithi006 ceph-mon[41734]: pgmap v1243: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:28:38.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:38 smithi090 ceph-mon[33646]: pgmap v1244: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:38.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:38 smithi006 ceph-mon[30108]: pgmap v1244: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:38.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:38 smithi006 ceph-mon[41734]: pgmap v1244: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:40.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:40 smithi090 ceph-mon[33646]: pgmap v1245: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:28:40.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:40 smithi006 ceph-mon[30108]: pgmap v1245: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:28:40.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:40 smithi006 ceph-mon[41734]: pgmap v1245: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:28:42.761 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:42 smithi006 ceph-mon[30108]: pgmap v1246: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:42.762 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:42 smithi006 ceph-mon[41734]: pgmap v1246: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:42.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:42 smithi090 ceph-mon[33646]: pgmap v1246: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:28:43.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:28:43 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:28:44.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:44 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:28:44.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:44 smithi090 ceph-mon[33646]: pgmap v1247: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:44.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:44 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:28:44.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:44 smithi006 ceph-mon[30108]: pgmap v1247: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:44.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:44 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:28:44.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:44 smithi006 ceph-mon[41734]: pgmap v1247: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:45.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:28:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:28:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:28:46.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:46 smithi090 ceph-mon[33646]: pgmap v1248: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:28:46.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:46 smithi006 ceph-mon[30108]: pgmap v1248: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:28:46.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:46 smithi006 ceph-mon[41734]: pgmap v1248: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:28:48.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:48 smithi090 ceph-mon[33646]: pgmap v1249: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:48.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:48 smithi006 ceph-mon[30108]: pgmap v1249: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:48.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:48 smithi006 ceph-mon[41734]: pgmap v1249: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:49.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:49 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:28:49.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:49 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:28:49.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:49 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:28:50.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:50 smithi090 ceph-mon[33646]: pgmap v1250: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:28:50.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:50 smithi006 ceph-mon[30108]: pgmap v1250: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:28:50.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:50 smithi006 ceph-mon[41734]: pgmap v1250: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:28:52.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:52 smithi090 ceph-mon[33646]: pgmap v1251: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:28:52.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:52 smithi006 ceph-mon[30108]: pgmap v1251: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:28:52.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:52 smithi006 ceph-mon[41734]: pgmap v1251: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:28:53.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:28:53 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:28:54.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:54 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:28:54.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:54 smithi090 ceph-mon[33646]: pgmap v1252: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:54.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:54 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:28:54.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:54 smithi006 ceph-mon[30108]: pgmap v1252: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:54.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:54 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:28:54.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:54 smithi006 ceph-mon[41734]: pgmap v1252: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:55.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:28:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:28:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:28:56.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:56 smithi090 ceph-mon[33646]: pgmap v1253: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:28:56.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:56 smithi006 ceph-mon[30108]: pgmap v1253: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:28:56.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:56 smithi006 ceph-mon[41734]: pgmap v1253: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:28:58.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:28:58 smithi090 ceph-mon[33646]: pgmap v1254: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:58.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:28:58 smithi006 ceph-mon[30108]: pgmap v1254: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:28:58.960 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:28:58 smithi006 ceph-mon[41734]: pgmap v1254: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:00.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:00 smithi090 ceph-mon[33646]: pgmap v1255: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:00.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:00 smithi006 ceph-mon[30108]: pgmap v1255: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:00.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:00 smithi006 ceph-mon[41734]: pgmap v1255: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:02.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:02 smithi090 ceph-mon[33646]: pgmap v1256: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:02.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:02 smithi006 ceph-mon[30108]: pgmap v1256: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:02.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:02 smithi006 ceph-mon[41734]: pgmap v1256: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:03.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:29:03 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:29:04.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:04 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:29:04.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:04 smithi090 ceph-mon[33646]: pgmap v1257: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:04.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:29:04.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:04 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:29:04.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:04 smithi006 ceph-mon[30108]: pgmap v1257: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:04.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:29:04.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:04 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:29:04.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:04 smithi006 ceph-mon[41734]: pgmap v1257: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:04.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:29:05.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:29:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:29:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:29:06.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:06 smithi090 ceph-mon[33646]: pgmap v1258: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:06.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:06 smithi006 ceph-mon[30108]: pgmap v1258: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:06.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:06 smithi006 ceph-mon[41734]: pgmap v1258: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:08.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:08 smithi090 ceph-mon[33646]: pgmap v1259: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:08.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:08 smithi006 ceph-mon[30108]: pgmap v1259: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:08.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:08 smithi006 ceph-mon[41734]: pgmap v1259: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:10.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:10 smithi090 ceph-mon[33646]: pgmap v1260: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:10.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:10 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:29:10.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:10 smithi006 ceph-mon[30108]: pgmap v1260: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:10.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:10 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:29:10.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:10 smithi006 ceph-mon[41734]: pgmap v1260: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:10.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:10 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:29:11.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:11 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:29:11.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:11 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:29:11.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:11 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:29:11.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:11 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:29:11.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:11 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:29:11.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:11 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:29:11.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:11 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:29:11.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:11 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:29:11.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:11 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:29:12.860 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:12 smithi006 ceph-mon[30108]: pgmap v1261: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:12.860 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:12 smithi006 ceph-mon[41734]: pgmap v1261: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:12.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:12 smithi090 ceph-mon[33646]: pgmap v1261: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:13.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:29:13 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:29:14.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:14 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:29:14.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:14 smithi090 ceph-mon[33646]: pgmap v1262: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:14.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:14 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:29:14.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:14 smithi006 ceph-mon[30108]: pgmap v1262: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:14.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:14 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:29:14.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:14 smithi006 ceph-mon[41734]: pgmap v1262: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:15.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:29:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:29:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:29:16.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:16 smithi090 ceph-mon[33646]: pgmap v1263: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:16.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:16 smithi006 ceph-mon[30108]: pgmap v1263: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:16.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:16 smithi006 ceph-mon[41734]: pgmap v1263: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:18.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:18 smithi090 ceph-mon[33646]: pgmap v1264: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:18.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:18 smithi006 ceph-mon[30108]: pgmap v1264: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:18.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:18 smithi006 ceph-mon[41734]: pgmap v1264: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:19.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:29:19.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:29:19.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:29:20.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:20 smithi090 ceph-mon[33646]: pgmap v1265: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:20.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:20 smithi006 ceph-mon[30108]: pgmap v1265: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:20.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:20 smithi006 ceph-mon[41734]: pgmap v1265: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:22.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:22 smithi090 ceph-mon[33646]: pgmap v1266: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:22.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:22 smithi006 ceph-mon[30108]: pgmap v1266: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:22.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:22 smithi006 ceph-mon[41734]: pgmap v1266: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:23.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:29:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:29:24.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:24 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:29:24.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:24 smithi090 ceph-mon[33646]: pgmap v1267: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:24.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:24 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:29:24.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:24 smithi006 ceph-mon[30108]: pgmap v1267: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:24.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:24 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:29:24.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:24 smithi006 ceph-mon[41734]: pgmap v1267: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:25.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:29:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:29:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:29:26.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:26 smithi090 ceph-mon[33646]: pgmap v1268: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:29:26.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:26 smithi006 ceph-mon[30108]: pgmap v1268: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:29:26.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:26 smithi006 ceph-mon[41734]: pgmap v1268: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:29:28.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:28 smithi090 ceph-mon[33646]: pgmap v1269: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:28.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:28 smithi006 ceph-mon[30108]: pgmap v1269: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:28.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:28 smithi006 ceph-mon[41734]: pgmap v1269: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:30.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:30 smithi090 ceph-mon[33646]: pgmap v1270: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:29:30.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:30 smithi006 ceph-mon[41734]: pgmap v1270: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:29:30.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:30 smithi006 ceph-mon[30108]: pgmap v1270: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:29:32.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:32 smithi090 ceph-mon[33646]: pgmap v1271: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:32.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:32 smithi006 ceph-mon[30108]: pgmap v1271: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:32.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:32 smithi006 ceph-mon[41734]: pgmap v1271: 228 pgs: 228 active+clean; 582 KiB data, 863 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:33.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:29:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:29:34.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:34 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:29:34.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:34 smithi090 ceph-mon[33646]: pgmap v1272: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:29:34.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:29:34.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:34 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:29:34.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:34 smithi006 ceph-mon[30108]: pgmap v1272: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:29:34.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:29:34.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:34 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:29:34.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:34 smithi006 ceph-mon[41734]: pgmap v1272: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:29:34.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:29:35.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:29:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:29:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:29:36.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:36 smithi090 ceph-mon[33646]: pgmap v1273: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:29:36.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:36 smithi006 ceph-mon[30108]: pgmap v1273: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:29:36.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:36 smithi006 ceph-mon[41734]: pgmap v1273: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:29:38.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:38 smithi090 ceph-mon[33646]: pgmap v1274: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:29:38.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:38 smithi006 ceph-mon[30108]: pgmap v1274: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:29:38.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:38 smithi006 ceph-mon[41734]: pgmap v1274: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:29:40.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:40 smithi090 ceph-mon[33646]: pgmap v1275: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:29:40.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:40 smithi006 ceph-mon[30108]: pgmap v1275: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:29:40.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:40 smithi006 ceph-mon[41734]: pgmap v1275: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:29:41.615 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.615 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.615 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.615 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.616 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.616 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.616 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.616 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.616 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.616 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.616 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.616 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.616 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.616 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.616 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.616 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.616 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.616 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.617 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.617 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.617 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.617 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.617 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.617 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.617 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.617 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.617 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.617 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.617 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.617 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.617 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.617 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.618 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.618 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.618 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.618 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.618 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.618 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.618 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.618 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.618 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.618 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.618 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.618 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.618 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.618 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.618 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.619 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.619 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.619 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.619 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.619 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.619 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.619 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.619 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.619 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.619 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.619 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.619 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.619 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.619 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.619 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.620 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.620 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.620 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.620 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.620 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.620 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.620 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.620 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.620 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.620 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.621 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.621 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.621 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.621 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.622 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.622 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.622 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.622 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.622 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.622 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.622 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.622 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.622 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.622 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.622 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.622 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.622 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.623 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.623 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.623 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.623 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.623 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.623 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.623 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.623 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.623 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.623 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:41.623 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:29:42.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:42 smithi090 ceph-mon[33646]: pgmap v1276: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:29:42.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:42 smithi006 ceph-mon[30108]: pgmap v1276: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:29:42.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:42 smithi006 ceph-mon[41734]: pgmap v1276: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:29:43.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:29:43 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:29:44.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:44 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:29:44.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:44 smithi090 ceph-mon[33646]: pgmap v1277: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:29:44.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:44 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:29:44.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:44 smithi006 ceph-mon[30108]: pgmap v1277: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:29:44.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:44 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:29:44.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:44 smithi006 ceph-mon[41734]: pgmap v1277: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:29:45.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:29:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:29:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:29:46.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:46 smithi090 ceph-mon[33646]: pgmap v1278: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:46.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:46 smithi006 ceph-mon[30108]: pgmap v1278: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:46.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:46 smithi006 ceph-mon[41734]: pgmap v1278: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:48.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:48 smithi090 ceph-mon[33646]: pgmap v1279: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:48.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:48 smithi006 ceph-mon[30108]: pgmap v1279: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:48.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:48 smithi006 ceph-mon[41734]: pgmap v1279: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:49.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:49 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:29:49.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:49 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:29:49.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:49 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:29:50.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:50 smithi090 ceph-mon[33646]: pgmap v1280: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:50.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:50 smithi006 ceph-mon[30108]: pgmap v1280: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:50.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:50 smithi006 ceph-mon[41734]: pgmap v1280: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:52.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:52 smithi090 ceph-mon[33646]: pgmap v1281: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:52.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:52 smithi006 ceph-mon[30108]: pgmap v1281: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:52.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:52 smithi006 ceph-mon[41734]: pgmap v1281: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:53.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:29:53 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:29:54.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:54 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:29:54.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:54 smithi090 ceph-mon[33646]: pgmap v1282: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:54.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:54 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:29:54.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:54 smithi006 ceph-mon[30108]: pgmap v1282: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:54.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:54 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:29:54.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:54 smithi006 ceph-mon[41734]: pgmap v1282: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:55.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:29:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:29:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:29:56.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:56 smithi090 ceph-mon[33646]: pgmap v1283: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:56.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:56 smithi006 ceph-mon[30108]: pgmap v1283: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:56.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:56 smithi006 ceph-mon[41734]: pgmap v1283: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:29:58.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:29:58 smithi090 ceph-mon[33646]: pgmap v1284: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:58.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:29:58 smithi006 ceph-mon[30108]: pgmap v1284: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:29:58.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:29:58 smithi006 ceph-mon[41734]: pgmap v1284: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:00.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:00 smithi090 ceph-mon[33646]: pgmap v1285: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:00.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:00 smithi090 ceph-mon[33646]: overall HEALTH_WARN 3 pool(s) do not have an application enabled 2024-03-24T12:30:00.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:00 smithi006 ceph-mon[30108]: pgmap v1285: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:00.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:00 smithi006 ceph-mon[30108]: overall HEALTH_WARN 3 pool(s) do not have an application enabled 2024-03-24T12:30:00.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:00 smithi006 ceph-mon[41734]: pgmap v1285: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:00.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:00 smithi006 ceph-mon[41734]: overall HEALTH_WARN 3 pool(s) do not have an application enabled 2024-03-24T12:30:02.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:02 smithi090 ceph-mon[33646]: pgmap v1286: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:02.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:02 smithi006 ceph-mon[30108]: pgmap v1286: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:02.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:02 smithi006 ceph-mon[41734]: pgmap v1286: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:03.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:30:03 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:30:04.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:04 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:30:04.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:04 smithi090 ceph-mon[33646]: pgmap v1287: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:04.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:30:04.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:04 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:30:04.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:04 smithi006 ceph-mon[30108]: pgmap v1287: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:04.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:30:04.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:04 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:30:04.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:04 smithi006 ceph-mon[41734]: pgmap v1287: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:04.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:30:05.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:30:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:30:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:30:06.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:06 smithi090 ceph-mon[33646]: pgmap v1288: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:06.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:06 smithi006 ceph-mon[30108]: pgmap v1288: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:06.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:06 smithi006 ceph-mon[41734]: pgmap v1288: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:08.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:08 smithi090 ceph-mon[33646]: pgmap v1289: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:08.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:08 smithi006 ceph-mon[30108]: pgmap v1289: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:08.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:08 smithi006 ceph-mon[41734]: pgmap v1289: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:10.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:10 smithi090 ceph-mon[33646]: pgmap v1290: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:10.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:10 smithi006 ceph-mon[30108]: pgmap v1290: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:10.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:10 smithi006 ceph-mon[41734]: pgmap v1290: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:11.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:11 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:30:11.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:11 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:30:11.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:11 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:30:12.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:12 smithi090 ceph-mon[33646]: pgmap v1291: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:12.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:12 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:30:12.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:12 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:30:12.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:12 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:30:12.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:12 smithi006 ceph-mon[30108]: pgmap v1291: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:12.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:12 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:30:12.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:12 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:30:12.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:12 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:30:12.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:12 smithi006 ceph-mon[41734]: pgmap v1291: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:12.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:12 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:30:12.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:12 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:30:12.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:12 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:30:13.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:30:13 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:30:14.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:14 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:30:14.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:14 smithi090 ceph-mon[33646]: pgmap v1292: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:14.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:14 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:30:14.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:14 smithi006 ceph-mon[30108]: pgmap v1292: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:14.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:14 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:30:14.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:14 smithi006 ceph-mon[41734]: pgmap v1292: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:15.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:30:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:30:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:30:16.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:16 smithi090 ceph-mon[33646]: pgmap v1293: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:16.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:16 smithi006 ceph-mon[30108]: pgmap v1293: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:16.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:16 smithi006 ceph-mon[41734]: pgmap v1293: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:18.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:18 smithi090 ceph-mon[33646]: pgmap v1294: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:18.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:18 smithi006 ceph-mon[30108]: pgmap v1294: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:18.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:18 smithi006 ceph-mon[41734]: pgmap v1294: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:19.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:30:19.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:30:19.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:30:20.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:20 smithi090 ceph-mon[33646]: pgmap v1295: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:20.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:20 smithi006 ceph-mon[30108]: pgmap v1295: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:20.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:20 smithi006 ceph-mon[41734]: pgmap v1295: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:22.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:22 smithi090 ceph-mon[33646]: pgmap v1296: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:22.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:22 smithi006 ceph-mon[30108]: pgmap v1296: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:22.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:22 smithi006 ceph-mon[41734]: pgmap v1296: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:23.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:30:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:30:24.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:24 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:30:24.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:24 smithi090 ceph-mon[33646]: pgmap v1297: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:24.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:24 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:30:24.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:24 smithi006 ceph-mon[30108]: pgmap v1297: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:24.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:24 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:30:24.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:24 smithi006 ceph-mon[41734]: pgmap v1297: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:25.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:30:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:30:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:30:26.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:26 smithi090 ceph-mon[33646]: pgmap v1298: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:26.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:26 smithi006 ceph-mon[30108]: pgmap v1298: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:26.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:26 smithi006 ceph-mon[41734]: pgmap v1298: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:28.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:28 smithi090 ceph-mon[33646]: pgmap v1299: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:28.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:28 smithi006 ceph-mon[30108]: pgmap v1299: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:28.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:28 smithi006 ceph-mon[41734]: pgmap v1299: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:30.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:30 smithi090 ceph-mon[33646]: pgmap v1300: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:30.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:30 smithi006 ceph-mon[30108]: pgmap v1300: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:30.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:30 smithi006 ceph-mon[41734]: pgmap v1300: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:32.426 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:32 smithi090 ceph-mon[33646]: pgmap v1301: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:32.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:32 smithi006 ceph-mon[30108]: pgmap v1301: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:32.459 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:32 smithi006 ceph-mon[41734]: pgmap v1301: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:33.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:30:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:30:34.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:34 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:30:34.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:34 smithi090 ceph-mon[33646]: pgmap v1302: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:34.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:30:34.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:34 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:30:34.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:34 smithi006 ceph-mon[30108]: pgmap v1302: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:34.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:30:34.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:34 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:30:34.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:34 smithi006 ceph-mon[41734]: pgmap v1302: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:34.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:30:35.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:30:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:30:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:30:36.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:36 smithi090 ceph-mon[33646]: pgmap v1303: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:36.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:36 smithi006 ceph-mon[30108]: pgmap v1303: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:36.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:36 smithi006 ceph-mon[41734]: pgmap v1303: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:38.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:38 smithi090 ceph-mon[33646]: pgmap v1304: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:38.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:38 smithi006 ceph-mon[30108]: pgmap v1304: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:38.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:38 smithi006 ceph-mon[41734]: pgmap v1304: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:40.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:40 smithi090 ceph-mon[33646]: pgmap v1305: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:40.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:40 smithi006 ceph-mon[30108]: pgmap v1305: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:40.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:40 smithi006 ceph-mon[41734]: pgmap v1305: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:42.761 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:42 smithi006 ceph-mon[30108]: pgmap v1306: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:42.761 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:42 smithi006 ceph-mon[41734]: pgmap v1306: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:42.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:42 smithi090 ceph-mon[33646]: pgmap v1306: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:43.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:30:43 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:30:44.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:44 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:30:44.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:44 smithi090 ceph-mon[33646]: pgmap v1307: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:44.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:44 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:30:44.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:44 smithi006 ceph-mon[30108]: pgmap v1307: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:44.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:44 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:30:44.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:44 smithi006 ceph-mon[41734]: pgmap v1307: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:45.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:30:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:30:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:30:46.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:46 smithi090 ceph-mon[33646]: pgmap v1308: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:46.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:46 smithi006 ceph-mon[30108]: pgmap v1308: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:46.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:46 smithi006 ceph-mon[41734]: pgmap v1308: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:48.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:48 smithi090 ceph-mon[33646]: pgmap v1309: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:48.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:48 smithi006 ceph-mon[30108]: pgmap v1309: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:48.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:48 smithi006 ceph-mon[41734]: pgmap v1309: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:49.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:49 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:30:49.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:49 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:30:49.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:49 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:30:50.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:50 smithi090 ceph-mon[33646]: pgmap v1310: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:30:50.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:50 smithi006 ceph-mon[30108]: pgmap v1310: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:30:50.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:50 smithi006 ceph-mon[41734]: pgmap v1310: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:30:52.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:52 smithi090 ceph-mon[33646]: pgmap v1311: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:52.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:52 smithi006 ceph-mon[30108]: pgmap v1311: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:52.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:52 smithi006 ceph-mon[41734]: pgmap v1311: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:30:53.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:30:53 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:30:54.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:54 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:30:54.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:54 smithi090 ceph-mon[33646]: pgmap v1312: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:54.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:54 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:30:54.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:54 smithi006 ceph-mon[30108]: pgmap v1312: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:54.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:54 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:30:54.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:54 smithi006 ceph-mon[41734]: pgmap v1312: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:55.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:30:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:30:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:30:56.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:56 smithi090 ceph-mon[33646]: pgmap v1313: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:30:56.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:56 smithi006 ceph-mon[30108]: pgmap v1313: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:30:56.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:56 smithi006 ceph-mon[41734]: pgmap v1313: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:30:58.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:30:58 smithi090 ceph-mon[33646]: pgmap v1314: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:58.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:30:58 smithi006 ceph-mon[30108]: pgmap v1314: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:30:58.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:30:58 smithi006 ceph-mon[41734]: pgmap v1314: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:00.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:00 smithi090 ceph-mon[33646]: pgmap v1315: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:31:00.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:00 smithi006 ceph-mon[30108]: pgmap v1315: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:31:00.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:00 smithi006 ceph-mon[41734]: pgmap v1315: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:31:02.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:02 smithi090 ceph-mon[33646]: pgmap v1316: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:02.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:02 smithi006 ceph-mon[30108]: pgmap v1316: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:02.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:02 smithi006 ceph-mon[41734]: pgmap v1316: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:03.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:31:03 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:31:04.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:04 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:31:04.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:04 smithi090 ceph-mon[33646]: pgmap v1317: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:04.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:31:04.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:04 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:31:04.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:04 smithi006 ceph-mon[30108]: pgmap v1317: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:04.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:31:04.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:04 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:31:04.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:04 smithi006 ceph-mon[41734]: pgmap v1317: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:04.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:31:05.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:31:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:31:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:31:06.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:06 smithi090 ceph-mon[33646]: pgmap v1318: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:31:06.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:06 smithi006 ceph-mon[30108]: pgmap v1318: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:31:06.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:06 smithi006 ceph-mon[41734]: pgmap v1318: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:31:08.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:08 smithi090 ceph-mon[33646]: pgmap v1319: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:08.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:08 smithi006 ceph-mon[30108]: pgmap v1319: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:08.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:08 smithi006 ceph-mon[41734]: pgmap v1319: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:10.176 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:31:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=cleanup t=2024-03-24T12:31:09.79260139Z level=info msg="Completed cleanup jobs" duration=6.341452ms 2024-03-24T12:31:10.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:10 smithi090 ceph-mon[33646]: pgmap v1320: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:10.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:10 smithi006 ceph-mon[30108]: pgmap v1320: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:10.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:10 smithi006 ceph-mon[41734]: pgmap v1320: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:12.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:12 smithi090 ceph-mon[33646]: pgmap v1321: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:12.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:12 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:31:12.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:12 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:31:12.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:12 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:31:12.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:12 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:31:12.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:12 smithi006 ceph-mon[30108]: pgmap v1321: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:12.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:12 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:31:12.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:12 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:31:12.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:12 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:31:12.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:12 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:31:12.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:12 smithi006 ceph-mon[41734]: pgmap v1321: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:12.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:12 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:31:12.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:12 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:31:12.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:12 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:31:12.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:12 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:31:13.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:31:13 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:31:14.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:14 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:31:14.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:14 smithi090 ceph-mon[33646]: pgmap v1322: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:14.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:14 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:31:14.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:14 smithi006 ceph-mon[30108]: pgmap v1322: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:14.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:14 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:31:14.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:14 smithi006 ceph-mon[41734]: pgmap v1322: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:15.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:31:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:31:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:31:16.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:16 smithi090 ceph-mon[33646]: pgmap v1323: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:16.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:16 smithi006 ceph-mon[30108]: pgmap v1323: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:16.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:16 smithi006 ceph-mon[41734]: pgmap v1323: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:18.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:18 smithi090 ceph-mon[33646]: pgmap v1324: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:18.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:18 smithi006 ceph-mon[30108]: pgmap v1324: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:18.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:18 smithi006 ceph-mon[41734]: pgmap v1324: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:19.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:31:19.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:31:19.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:31:20.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:20 smithi090 ceph-mon[33646]: pgmap v1325: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:20.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:20 smithi006 ceph-mon[30108]: pgmap v1325: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:20.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:20 smithi006 ceph-mon[41734]: pgmap v1325: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:21.627 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.627 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.627 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.627 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.627 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.627 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.627 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.627 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.627 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.627 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.627 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.628 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.628 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.628 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.628 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.628 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.628 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.628 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.628 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.628 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.628 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.628 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.628 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.628 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.629 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.629 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.629 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.629 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.629 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.629 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.629 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.629 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.629 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.629 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.629 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.629 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.629 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.629 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.630 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.630 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.630 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.630 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.630 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.630 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.630 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.630 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.630 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.630 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.630 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.630 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.630 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.630 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.631 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.631 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.631 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.631 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.631 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.631 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.631 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.631 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.631 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.631 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.631 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.631 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.631 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.631 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.632 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.632 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.632 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.632 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.632 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.632 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.632 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.632 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.632 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.632 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.632 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.632 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.632 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.632 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.632 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.633 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.633 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.633 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.633 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.633 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.633 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.633 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.633 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.633 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.633 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.633 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.633 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.633 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.633 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.633 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.634 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.634 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.634 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:21.634 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:31:22.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:22 smithi090 ceph-mon[33646]: pgmap v1326: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:22.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:22 smithi006 ceph-mon[30108]: pgmap v1326: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:22.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:22 smithi006 ceph-mon[41734]: pgmap v1326: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:23.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:31:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:31:24.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:24 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:31:24.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:24 smithi090 ceph-mon[33646]: pgmap v1327: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:24.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:24 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:31:24.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:24 smithi006 ceph-mon[30108]: pgmap v1327: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:24.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:24 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:31:24.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:24 smithi006 ceph-mon[41734]: pgmap v1327: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:25.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:31:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:31:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:31:26.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:26 smithi090 ceph-mon[33646]: pgmap v1328: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:26.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:26 smithi006 ceph-mon[30108]: pgmap v1328: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:26.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:26 smithi006 ceph-mon[41734]: pgmap v1328: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:28.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:28 smithi090 ceph-mon[33646]: pgmap v1329: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:28.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:28 smithi006 ceph-mon[30108]: pgmap v1329: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:28.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:28 smithi006 ceph-mon[41734]: pgmap v1329: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:30.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:30 smithi090 ceph-mon[33646]: pgmap v1330: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:30.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:30 smithi006 ceph-mon[30108]: pgmap v1330: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:30.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:30 smithi006 ceph-mon[41734]: pgmap v1330: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:32.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:32 smithi090 ceph-mon[33646]: pgmap v1331: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:32.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:32 smithi006 ceph-mon[30108]: pgmap v1331: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:32.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:32 smithi006 ceph-mon[41734]: pgmap v1331: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:33.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:31:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:31:34.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:34 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:31:34.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:34 smithi090 ceph-mon[33646]: pgmap v1332: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:34.927 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:31:34.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:34 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:31:34.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:34 smithi006 ceph-mon[30108]: pgmap v1332: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:34.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:31:34.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:34 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:31:34.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:34 smithi006 ceph-mon[41734]: pgmap v1332: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:34.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:31:35.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:31:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:31:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:31:36.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:36 smithi090 ceph-mon[33646]: pgmap v1333: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:36.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:36 smithi006 ceph-mon[30108]: pgmap v1333: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:36.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:36 smithi006 ceph-mon[41734]: pgmap v1333: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:38.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:38 smithi090 ceph-mon[33646]: pgmap v1334: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:38.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:38 smithi006 ceph-mon[30108]: pgmap v1334: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:38.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:38 smithi006 ceph-mon[41734]: pgmap v1334: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:40.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:40 smithi090 ceph-mon[33646]: pgmap v1335: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:31:40.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:40 smithi006 ceph-mon[30108]: pgmap v1335: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:31:40.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:40 smithi006 ceph-mon[41734]: pgmap v1335: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:31:42.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:42 smithi090 ceph-mon[33646]: pgmap v1336: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:42.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:42 smithi006 ceph-mon[30108]: pgmap v1336: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:42.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:42 smithi006 ceph-mon[41734]: pgmap v1336: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:43.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:31:43 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:31:44.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:44 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:31:44.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:44 smithi090 ceph-mon[33646]: pgmap v1337: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:44.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:44 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:31:44.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:44 smithi006 ceph-mon[30108]: pgmap v1337: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:44.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:44 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:31:44.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:44 smithi006 ceph-mon[41734]: pgmap v1337: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:45.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:31:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:31:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:31:46.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:46 smithi090 ceph-mon[33646]: pgmap v1338: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:31:46.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:46 smithi006 ceph-mon[30108]: pgmap v1338: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:31:46.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:46 smithi006 ceph-mon[41734]: pgmap v1338: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:31:48.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:48 smithi090 ceph-mon[33646]: pgmap v1339: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:48.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:48 smithi006 ceph-mon[30108]: pgmap v1339: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:48.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:48 smithi006 ceph-mon[41734]: pgmap v1339: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:49.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:49 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:31:49.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:49 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:31:49.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:49 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:31:50.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:50 smithi090 ceph-mon[33646]: pgmap v1340: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:31:50.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:50 smithi006 ceph-mon[30108]: pgmap v1340: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:31:50.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:50 smithi006 ceph-mon[41734]: pgmap v1340: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:31:52.426 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:52 smithi090 ceph-mon[33646]: pgmap v1341: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:52.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:52 smithi006 ceph-mon[30108]: pgmap v1341: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:52.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:52 smithi006 ceph-mon[41734]: pgmap v1341: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:53.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:31:53 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:31:54.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:54 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:31:54.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:54 smithi090 ceph-mon[33646]: pgmap v1342: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:54.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:54 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:31:54.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:54 smithi006 ceph-mon[30108]: pgmap v1342: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:54.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:54 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:31:54.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:54 smithi006 ceph-mon[41734]: pgmap v1342: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:55.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:31:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:31:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:31:56.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:56 smithi090 ceph-mon[33646]: pgmap v1343: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:56.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:56 smithi006 ceph-mon[30108]: pgmap v1343: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:56.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:56 smithi006 ceph-mon[41734]: pgmap v1343: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:31:58.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:31:58 smithi090 ceph-mon[33646]: pgmap v1344: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:58.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:31:58 smithi006 ceph-mon[30108]: pgmap v1344: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:31:58.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:31:58 smithi006 ceph-mon[41734]: pgmap v1344: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:00.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:00 smithi090 ceph-mon[33646]: pgmap v1345: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:00.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:00 smithi006 ceph-mon[30108]: pgmap v1345: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:00.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:00 smithi006 ceph-mon[41734]: pgmap v1345: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:02.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:02 smithi090 ceph-mon[33646]: pgmap v1346: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:02.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:02 smithi006 ceph-mon[30108]: pgmap v1346: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:02.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:02 smithi006 ceph-mon[41734]: pgmap v1346: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:03.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:32:03 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:32:04.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:04 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:32:04.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:04 smithi090 ceph-mon[33646]: pgmap v1347: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:04.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:32:04.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:04 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:32:04.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:04 smithi006 ceph-mon[30108]: pgmap v1347: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:04.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:32:04.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:04 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:32:04.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:04 smithi006 ceph-mon[41734]: pgmap v1347: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:04.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:32:05.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:32:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:32:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:32:06.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:06 smithi090 ceph-mon[33646]: pgmap v1348: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:06.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:06 smithi006 ceph-mon[30108]: pgmap v1348: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:06.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:06 smithi006 ceph-mon[41734]: pgmap v1348: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:08.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:08 smithi090 ceph-mon[33646]: pgmap v1349: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:08.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:08 smithi006 ceph-mon[30108]: pgmap v1349: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:08.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:08 smithi006 ceph-mon[41734]: pgmap v1349: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:10.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:10 smithi090 ceph-mon[33646]: pgmap v1350: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:10.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:10 smithi006 ceph-mon[30108]: pgmap v1350: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:10.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:10 smithi006 ceph-mon[41734]: pgmap v1350: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:12.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:12 smithi090 ceph-mon[33646]: pgmap v1351: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:12.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:12 smithi006 ceph-mon[30108]: pgmap v1351: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:12.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:12 smithi006 ceph-mon[41734]: pgmap v1351: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:13.528 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:32:13 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:32:13.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:13 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:32:13.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:13 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:32:13.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:13 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:32:13.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:13 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:32:13.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:13 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:32:13.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:13 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:32:13.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:13 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:32:13.962 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:13 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:32:13.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:13 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:32:13.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:13 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:32:13.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:13 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:32:13.962 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:13 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:32:14.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:14 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:32:14.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:14 smithi090 ceph-mon[33646]: pgmap v1352: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:14.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:14 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:32:14.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:14 smithi006 ceph-mon[30108]: pgmap v1352: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:14.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:14 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:32:14.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:14 smithi006 ceph-mon[41734]: pgmap v1352: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:15.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:32:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:32:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:32:16.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:16 smithi090 ceph-mon[33646]: pgmap v1353: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:16.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:16 smithi006 ceph-mon[30108]: pgmap v1353: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:16.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:16 smithi006 ceph-mon[41734]: pgmap v1353: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:18.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:18 smithi090 ceph-mon[33646]: pgmap v1354: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:18.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:18 smithi006 ceph-mon[30108]: pgmap v1354: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:18.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:18 smithi006 ceph-mon[41734]: pgmap v1354: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:19.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:32:19.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:32:19.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:32:20.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:20 smithi090 ceph-mon[33646]: pgmap v1355: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:20.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:20 smithi006 ceph-mon[30108]: pgmap v1355: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:20.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:20 smithi006 ceph-mon[41734]: pgmap v1355: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:22.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:22 smithi090 ceph-mon[33646]: pgmap v1356: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:22.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:22 smithi006 ceph-mon[30108]: pgmap v1356: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:22.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:22 smithi006 ceph-mon[41734]: pgmap v1356: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:23.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:32:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:32:24.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:24 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:32:24.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:24 smithi090 ceph-mon[33646]: pgmap v1357: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:24.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:24 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:32:24.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:24 smithi006 ceph-mon[30108]: pgmap v1357: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:24.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:24 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:32:24.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:24 smithi006 ceph-mon[41734]: pgmap v1357: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:25.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:32:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:32:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:32:26.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:26 smithi090 ceph-mon[33646]: pgmap v1358: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:32:26.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:26 smithi006 ceph-mon[30108]: pgmap v1358: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:32:26.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:26 smithi006 ceph-mon[41734]: pgmap v1358: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:32:28.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:28 smithi090 ceph-mon[33646]: pgmap v1359: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:28.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:28 smithi006 ceph-mon[30108]: pgmap v1359: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:28.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:28 smithi006 ceph-mon[41734]: pgmap v1359: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:30.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:30 smithi090 ceph-mon[33646]: pgmap v1360: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:32:30.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:30 smithi006 ceph-mon[30108]: pgmap v1360: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:32:30.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:30 smithi006 ceph-mon[41734]: pgmap v1360: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:32:32.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:32 smithi090 ceph-mon[33646]: pgmap v1361: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:32.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:32 smithi006 ceph-mon[30108]: pgmap v1361: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:32.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:32 smithi006 ceph-mon[41734]: pgmap v1361: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:33.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:32:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:32:34.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:34 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:32:34.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:34 smithi090 ceph-mon[33646]: pgmap v1362: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:34.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:32:34.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:34 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:32:34.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:34 smithi006 ceph-mon[30108]: pgmap v1362: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:34.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:32:34.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:34 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:32:34.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:34 smithi006 ceph-mon[41734]: pgmap v1362: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:34.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:32:35.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:32:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:32:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:32:36.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:36 smithi090 ceph-mon[33646]: pgmap v1363: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:32:36.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:36 smithi006 ceph-mon[30108]: pgmap v1363: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:32:36.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:36 smithi006 ceph-mon[41734]: pgmap v1363: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:32:38.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:38 smithi090 ceph-mon[33646]: pgmap v1364: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:38.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:38 smithi006 ceph-mon[30108]: pgmap v1364: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:38.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:38 smithi006 ceph-mon[41734]: pgmap v1364: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:40.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:40 smithi090 ceph-mon[33646]: pgmap v1365: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:32:40.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:40 smithi006 ceph-mon[30108]: pgmap v1365: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:32:40.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:40 smithi006 ceph-mon[41734]: pgmap v1365: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:32:42.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:42 smithi090 ceph-mon[33646]: pgmap v1366: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:42.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:42 smithi006 ceph-mon[30108]: pgmap v1366: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:42.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:42 smithi006 ceph-mon[41734]: pgmap v1366: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:43.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:32:43 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:32:44.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:44 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:32:44.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:44 smithi090 ceph-mon[33646]: pgmap v1367: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:44.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:44 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:32:44.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:44 smithi006 ceph-mon[30108]: pgmap v1367: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:44.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:44 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:32:44.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:44 smithi006 ceph-mon[41734]: pgmap v1367: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:45.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:32:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:32:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:32:46.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:46 smithi090 ceph-mon[33646]: pgmap v1368: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:32:46.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:46 smithi006 ceph-mon[30108]: pgmap v1368: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:32:46.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:46 smithi006 ceph-mon[41734]: pgmap v1368: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:32:48.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:48 smithi090 ceph-mon[33646]: pgmap v1369: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:48.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:48 smithi006 ceph-mon[30108]: pgmap v1369: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:48.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:48 smithi006 ceph-mon[41734]: pgmap v1369: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:49.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:49 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:32:49.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:49 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:32:49.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:49 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:32:50.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:50 smithi090 ceph-mon[33646]: pgmap v1370: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:50.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:50 smithi006 ceph-mon[30108]: pgmap v1370: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:50.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:50 smithi006 ceph-mon[41734]: pgmap v1370: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:52.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:52 smithi090 ceph-mon[33646]: pgmap v1371: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:52.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:52 smithi006 ceph-mon[30108]: pgmap v1371: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:52.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:52 smithi006 ceph-mon[41734]: pgmap v1371: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:32:53.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:32:53 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:32:54.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:54 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:32:54.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:54 smithi090 ceph-mon[33646]: pgmap v1372: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:54.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:54 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:32:54.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:54 smithi006 ceph-mon[30108]: pgmap v1372: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:54.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:54 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:32:54.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:54 smithi006 ceph-mon[41734]: pgmap v1372: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:55.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:32:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:32:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:32:56.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:56 smithi090 ceph-mon[33646]: pgmap v1373: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:32:56.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:56 smithi006 ceph-mon[30108]: pgmap v1373: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:32:56.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:56 smithi006 ceph-mon[41734]: pgmap v1373: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:32:58.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:32:58 smithi090 ceph-mon[33646]: pgmap v1374: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:58.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:32:58 smithi006 ceph-mon[30108]: pgmap v1374: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:32:58.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:32:58 smithi006 ceph-mon[41734]: pgmap v1374: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:00.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:00 smithi090 ceph-mon[33646]: pgmap v1375: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:00.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:00 smithi006 ceph-mon[30108]: pgmap v1375: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:00.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:00 smithi006 ceph-mon[41734]: pgmap v1375: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:01.638 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.639 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.639 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.639 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.639 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.639 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.639 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.639 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.639 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.639 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.639 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.639 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.639 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.640 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.640 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.640 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.640 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.640 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.640 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.640 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.640 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.640 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.640 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.640 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.640 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.640 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.640 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.640 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.640 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.640 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.641 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.642 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.643 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.644 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.645 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.645 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.645 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:01.645 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:33:02.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:02 smithi090 ceph-mon[33646]: pgmap v1376: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:02.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:02 smithi006 ceph-mon[30108]: pgmap v1376: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:02.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:02 smithi006 ceph-mon[41734]: pgmap v1376: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:03.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:33:03 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:33:04.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:04 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:33:04.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:04 smithi090 ceph-mon[33646]: pgmap v1377: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:04.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:33:04.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:04 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:33:04.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:04 smithi006 ceph-mon[30108]: pgmap v1377: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:04.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:33:04.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:04 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:33:04.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:04 smithi006 ceph-mon[41734]: pgmap v1377: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:04.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:33:05.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:33:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:33:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:33:06.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:06 smithi090 ceph-mon[33646]: pgmap v1378: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:33:06.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:06 smithi006 ceph-mon[30108]: pgmap v1378: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:33:06.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:06 smithi006 ceph-mon[41734]: pgmap v1378: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:33:08.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:08 smithi090 ceph-mon[33646]: pgmap v1379: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:08.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:08 smithi006 ceph-mon[30108]: pgmap v1379: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:08.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:08 smithi006 ceph-mon[41734]: pgmap v1379: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:10.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:10 smithi090 ceph-mon[33646]: pgmap v1380: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:10.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:10 smithi006 ceph-mon[30108]: pgmap v1380: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:10.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:10 smithi006 ceph-mon[41734]: pgmap v1380: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:12.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:12 smithi090 ceph-mon[33646]: pgmap v1381: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:12.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:12 smithi006 ceph-mon[30108]: pgmap v1381: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:12.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:12 smithi006 ceph-mon[41734]: pgmap v1381: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:13.639 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:33:13 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:33:13.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:13 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:33:13.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:13 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:33:13.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:13 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:33:14.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:14 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:33:14.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:14 smithi090 ceph-mon[33646]: pgmap v1382: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:14.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:14 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:33:14.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:14 smithi006 ceph-mon[41734]: pgmap v1382: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:14.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:14 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:33:14.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:14 smithi006 ceph-mon[30108]: pgmap v1382: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:15.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:33:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:33:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:33:16.792 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:16 smithi090 ceph-mon[33646]: pgmap v1383: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:16.792 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:16 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:33:16.792 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:16 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:33:16.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:16 smithi006 ceph-mon[30108]: pgmap v1383: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:16.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:16 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:33:16.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:16 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:33:16.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:16 smithi006 ceph-mon[41734]: pgmap v1383: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:16.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:16 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:33:16.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:16 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:33:18.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:18 smithi090 ceph-mon[33646]: pgmap v1384: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:18.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:18 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:33:18.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:18 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:33:18.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:18 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:33:18.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:18 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:33:18.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:18 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:33:18.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:18 smithi006 ceph-mon[30108]: pgmap v1384: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:18.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:18 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:33:18.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:18 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:33:18.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:18 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:33:18.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:18 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:33:18.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:18 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:33:18.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:18 smithi006 ceph-mon[41734]: pgmap v1384: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:18.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:18 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:33:18.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:18 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:33:18.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:18 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:33:18.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:18 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:33:18.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:18 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:33:19.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:33:19.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:33:19.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:33:20.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:20 smithi090 ceph-mon[33646]: pgmap v1385: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:20.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:20 smithi006 ceph-mon[30108]: pgmap v1385: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:20.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:20 smithi006 ceph-mon[41734]: pgmap v1385: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:22.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:22 smithi090 ceph-mon[33646]: pgmap v1386: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:22.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:22 smithi006 ceph-mon[30108]: pgmap v1386: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:22.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:22 smithi006 ceph-mon[41734]: pgmap v1386: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:23.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:33:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:33:24.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:24 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:33:24.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:24 smithi090 ceph-mon[33646]: pgmap v1387: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:24.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:24 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:33:24.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:24 smithi006 ceph-mon[41734]: pgmap v1387: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:24.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:24 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:33:24.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:24 smithi006 ceph-mon[30108]: pgmap v1387: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:25.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:33:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:33:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:33:26.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:26 smithi090 ceph-mon[33646]: pgmap v1388: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:26.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:26 smithi006 ceph-mon[30108]: pgmap v1388: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:26.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:26 smithi006 ceph-mon[41734]: pgmap v1388: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:28.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:28 smithi090 ceph-mon[33646]: pgmap v1389: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:28.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:28 smithi006 ceph-mon[30108]: pgmap v1389: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:28.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:28 smithi006 ceph-mon[41734]: pgmap v1389: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:30.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:30 smithi090 ceph-mon[33646]: pgmap v1390: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:30.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:30 smithi006 ceph-mon[30108]: pgmap v1390: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:30.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:30 smithi006 ceph-mon[41734]: pgmap v1390: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:32.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:32 smithi090 ceph-mon[33646]: pgmap v1391: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:32.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:32 smithi006 ceph-mon[30108]: pgmap v1391: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:32.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:32 smithi006 ceph-mon[41734]: pgmap v1391: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:33.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:33:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:33:34.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:34 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:33:34.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:34 smithi090 ceph-mon[33646]: pgmap v1392: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:34.926 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:33:34.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:34 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:33:34.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:34 smithi006 ceph-mon[30108]: pgmap v1392: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:34.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:33:34.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:34 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:33:34.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:34 smithi006 ceph-mon[41734]: pgmap v1392: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:34.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:33:35.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:33:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:33:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:33:36.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:36 smithi006 ceph-mon[30108]: pgmap v1393: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:33:36.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:36 smithi006 ceph-mon[41734]: pgmap v1393: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:33:37.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:36 smithi090 ceph-mon[33646]: pgmap v1393: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:33:38.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:38 smithi006 ceph-mon[30108]: pgmap v1394: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:38.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:38 smithi006 ceph-mon[41734]: pgmap v1394: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:39.006 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:38 smithi090 ceph-mon[33646]: pgmap v1394: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:40.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:40 smithi006 ceph-mon[30108]: pgmap v1395: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:33:40.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:40 smithi006 ceph-mon[41734]: pgmap v1395: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:33:41.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:40 smithi090 ceph-mon[33646]: pgmap v1395: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:33:42.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:42 smithi006 ceph-mon[30108]: pgmap v1396: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:42.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:42 smithi006 ceph-mon[41734]: pgmap v1396: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:43.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:42 smithi090 ceph-mon[33646]: pgmap v1396: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:43.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:33:43 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:33:44.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:44 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:33:44.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:44 smithi006 ceph-mon[30108]: pgmap v1397: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:44.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:44 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:33:44.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:44 smithi006 ceph-mon[41734]: pgmap v1397: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:45.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:44 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:33:45.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:44 smithi090 ceph-mon[33646]: pgmap v1397: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:45.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:33:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:33:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:33:46.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:46 smithi006 ceph-mon[30108]: pgmap v1398: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:46.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:46 smithi006 ceph-mon[41734]: pgmap v1398: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:47.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:46 smithi090 ceph-mon[33646]: pgmap v1398: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:48.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:48 smithi006 ceph-mon[30108]: pgmap v1399: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:48.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:48 smithi006 ceph-mon[41734]: pgmap v1399: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:49.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:48 smithi090 ceph-mon[33646]: pgmap v1399: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:49.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:49 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:33:49.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:49 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:33:50.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:49 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:33:50.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:50 smithi006 ceph-mon[30108]: pgmap v1400: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:50.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:50 smithi006 ceph-mon[41734]: pgmap v1400: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:51.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:50 smithi090 ceph-mon[33646]: pgmap v1400: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:52.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:52 smithi006 ceph-mon[30108]: pgmap v1401: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:52.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:52 smithi006 ceph-mon[41734]: pgmap v1401: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:53.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:52 smithi090 ceph-mon[33646]: pgmap v1401: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:53.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:33:53 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:33:54.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:54 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:33:54.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:54 smithi006 ceph-mon[30108]: pgmap v1402: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:54.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:54 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:33:54.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:54 smithi006 ceph-mon[41734]: pgmap v1402: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:55.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:54 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:33:55.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:54 smithi090 ceph-mon[33646]: pgmap v1402: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:55.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:33:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:33:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:33:56.977 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:56 smithi090 ceph-mon[33646]: pgmap v1403: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:57.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:56 smithi006 ceph-mon[30108]: pgmap v1403: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:57.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:56 smithi006 ceph-mon[41734]: pgmap v1403: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:33:59.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:33:58 smithi090 ceph-mon[33646]: pgmap v1404: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:59.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:33:58 smithi006 ceph-mon[30108]: pgmap v1404: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:33:59.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:33:58 smithi006 ceph-mon[41734]: pgmap v1404: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:01.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:00 smithi090 ceph-mon[33646]: pgmap v1405: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:01.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:00 smithi006 ceph-mon[30108]: pgmap v1405: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:01.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:00 smithi006 ceph-mon[41734]: pgmap v1405: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:03.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:02 smithi090 ceph-mon[33646]: pgmap v1406: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:03.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:02 smithi006 ceph-mon[30108]: pgmap v1406: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:03.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:02 smithi006 ceph-mon[41734]: pgmap v1406: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:03.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:34:03 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:34:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:04 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:34:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:04 smithi090 ceph-mon[33646]: pgmap v1407: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:34:05.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:04 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:34:05.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:04 smithi006 ceph-mon[30108]: pgmap v1407: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:05.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:34:05.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:04 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:34:05.209 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:04 smithi006 ceph-mon[41734]: pgmap v1407: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:05.209 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:34:05.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:34:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:34:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:34:07.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:06 smithi090 ceph-mon[33646]: pgmap v1408: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:07.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:06 smithi006 ceph-mon[30108]: pgmap v1408: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:07.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:06 smithi006 ceph-mon[41734]: pgmap v1408: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:09.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:08 smithi090 ceph-mon[33646]: pgmap v1409: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:09.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:08 smithi006 ceph-mon[30108]: pgmap v1409: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:09.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:08 smithi006 ceph-mon[41734]: pgmap v1409: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:11.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:10 smithi090 ceph-mon[33646]: pgmap v1410: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:11.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:10 smithi006 ceph-mon[30108]: pgmap v1410: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:11.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:10 smithi006 ceph-mon[41734]: pgmap v1410: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:13.010 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:12 smithi006 ceph-mon[30108]: pgmap v1411: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:13.011 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:12 smithi006 ceph-mon[41734]: pgmap v1411: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:13.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:12 smithi090 ceph-mon[33646]: pgmap v1411: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:13.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:34:13 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:34:15.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:14 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:34:15.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:14 smithi090 ceph-mon[33646]: pgmap v1412: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:15.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:14 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:34:15.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:14 smithi006 ceph-mon[30108]: pgmap v1412: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:15.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:14 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:34:15.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:14 smithi006 ceph-mon[41734]: pgmap v1412: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:15.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:34:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:34:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:34:17.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:16 smithi090 ceph-mon[33646]: pgmap v1413: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:34:17.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:16 smithi006 ceph-mon[30108]: pgmap v1413: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:34:17.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:16 smithi006 ceph-mon[41734]: pgmap v1413: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:34:19.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:18 smithi090 ceph-mon[33646]: pgmap v1414: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:19.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:18 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:34:19.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:18 smithi006 ceph-mon[30108]: pgmap v1414: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:19.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:18 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:34:19.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:18 smithi006 ceph-mon[41734]: pgmap v1414: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:19.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:18 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:34:20.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:34:20.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:34:20.176 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:19 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:34:20.176 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:34:20.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:34:20.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:34:20.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:19 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:34:20.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:34:20.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:34:20.209 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:34:20.209 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:19 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:34:20.209 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:34:21.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:20 smithi090 ceph-mon[33646]: pgmap v1415: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:21.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:20 smithi006 ceph-mon[30108]: pgmap v1415: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:21.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:20 smithi006 ceph-mon[41734]: pgmap v1415: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:23.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:22 smithi090 ceph-mon[33646]: pgmap v1416: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:23.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:22 smithi006 ceph-mon[41734]: pgmap v1416: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:23.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:22 smithi006 ceph-mon[30108]: pgmap v1416: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:23.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:34:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:34:25.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:24 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:34:25.176 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:24 smithi090 ceph-mon[33646]: pgmap v1417: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:25.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:24 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:34:25.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:24 smithi006 ceph-mon[30108]: pgmap v1417: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:25.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:24 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:34:25.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:24 smithi006 ceph-mon[41734]: pgmap v1417: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:25.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:34:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:34:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:34:27.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:26 smithi090 ceph-mon[33646]: pgmap v1418: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:27.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:26 smithi006 ceph-mon[30108]: pgmap v1418: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:27.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:26 smithi006 ceph-mon[41734]: pgmap v1418: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:29.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:28 smithi090 ceph-mon[33646]: pgmap v1419: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:29.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:28 smithi006 ceph-mon[30108]: pgmap v1419: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:29.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:28 smithi006 ceph-mon[41734]: pgmap v1419: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:31.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:30 smithi090 ceph-mon[33646]: pgmap v1420: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:31.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:30 smithi006 ceph-mon[30108]: pgmap v1420: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:31.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:30 smithi006 ceph-mon[41734]: pgmap v1420: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:33.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:32 smithi090 ceph-mon[33646]: pgmap v1421: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:33.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:32 smithi006 ceph-mon[30108]: pgmap v1421: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:33.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:32 smithi006 ceph-mon[41734]: pgmap v1421: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:33.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:34:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:34:35.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:34 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:34:35.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:34 smithi090 ceph-mon[33646]: pgmap v1422: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:35.176 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:34:35.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:34 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:34:35.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:34 smithi006 ceph-mon[30108]: pgmap v1422: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:35.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:34:35.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:34 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:34:35.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:34 smithi006 ceph-mon[41734]: pgmap v1422: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:35.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:34:35.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:34:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:34:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:34:37.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:36 smithi090 ceph-mon[33646]: pgmap v1423: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:37.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:36 smithi006 ceph-mon[30108]: pgmap v1423: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:37.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:36 smithi006 ceph-mon[41734]: pgmap v1423: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:39.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:38 smithi090 ceph-mon[33646]: pgmap v1424: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:39.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:38 smithi006 ceph-mon[30108]: pgmap v1424: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:39.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:38 smithi006 ceph-mon[41734]: pgmap v1424: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:41.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:40 smithi090 ceph-mon[33646]: pgmap v1425: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:41.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:40 smithi006 ceph-mon[30108]: pgmap v1425: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:41.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:40 smithi006 ceph-mon[41734]: pgmap v1425: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:41.650 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.650 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.651 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.651 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.651 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.651 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.651 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.651 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.651 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.651 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.651 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.651 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.651 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.651 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.651 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.652 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.652 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.652 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.652 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.652 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.652 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.652 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.652 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.652 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.652 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.652 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.652 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.652 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.652 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.653 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.653 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.653 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.653 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.653 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.653 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.653 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.653 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.653 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.653 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.653 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.653 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.653 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.653 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.653 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.654 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.654 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.654 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.654 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.654 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.654 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.654 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.654 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.654 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.654 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.654 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.654 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.654 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.654 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.654 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.655 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.655 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.655 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.655 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.655 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.655 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.655 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.655 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.655 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.655 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.655 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.655 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.655 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.655 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.656 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.656 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.656 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.656 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.656 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.656 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.656 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.656 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.656 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.656 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.656 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.656 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.656 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.656 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.656 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.657 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.657 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.657 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.657 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.657 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.657 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.657 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.657 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.657 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.657 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.657 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:41.657 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:34:43.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:42 smithi090 ceph-mon[33646]: pgmap v1426: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:43.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:42 smithi006 ceph-mon[30108]: pgmap v1426: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:43.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:42 smithi006 ceph-mon[41734]: pgmap v1426: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:43.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:34:43 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:34:45.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:44 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:34:45.176 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:44 smithi090 ceph-mon[33646]: pgmap v1427: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:45.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:44 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:34:45.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:44 smithi006 ceph-mon[30108]: pgmap v1427: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:45.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:44 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:34:45.209 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:44 smithi006 ceph-mon[41734]: pgmap v1427: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:45.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:34:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:34:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:34:47.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:46 smithi090 ceph-mon[33646]: pgmap v1428: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:34:47.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:46 smithi006 ceph-mon[30108]: pgmap v1428: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:34:47.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:46 smithi006 ceph-mon[41734]: pgmap v1428: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:34:49.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:48 smithi090 ceph-mon[33646]: pgmap v1429: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:49.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:48 smithi006 ceph-mon[30108]: pgmap v1429: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:49.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:48 smithi006 ceph-mon[41734]: pgmap v1429: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:50.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:49 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:34:50.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:49 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:34:50.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:49 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:34:51.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:50 smithi090 ceph-mon[33646]: pgmap v1430: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:34:51.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:50 smithi006 ceph-mon[30108]: pgmap v1430: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:34:51.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:50 smithi006 ceph-mon[41734]: pgmap v1430: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:34:53.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:52 smithi090 ceph-mon[33646]: pgmap v1431: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:34:53.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:52 smithi006 ceph-mon[30108]: pgmap v1431: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:34:53.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:52 smithi006 ceph-mon[41734]: pgmap v1431: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:34:53.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:34:53 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:34:55.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:54 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:34:55.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:54 smithi090 ceph-mon[33646]: pgmap v1432: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:55.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:54 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:34:55.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:54 smithi006 ceph-mon[30108]: pgmap v1432: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:55.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:54 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:34:55.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:54 smithi006 ceph-mon[41734]: pgmap v1432: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:55.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:34:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:34:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:34:57.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:56 smithi090 ceph-mon[33646]: pgmap v1433: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:57.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:56 smithi006 ceph-mon[30108]: pgmap v1433: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:57.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:56 smithi006 ceph-mon[41734]: pgmap v1433: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:34:59.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:34:58 smithi090 ceph-mon[33646]: pgmap v1434: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:59.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:34:58 smithi006 ceph-mon[30108]: pgmap v1434: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:34:59.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:34:58 smithi006 ceph-mon[41734]: pgmap v1434: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:01.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:00 smithi090 ceph-mon[33646]: pgmap v1435: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:01.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:00 smithi006 ceph-mon[30108]: pgmap v1435: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:01.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:00 smithi006 ceph-mon[41734]: pgmap v1435: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:03.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:02 smithi090 ceph-mon[33646]: pgmap v1436: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:03.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:02 smithi006 ceph-mon[30108]: pgmap v1436: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:03.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:02 smithi006 ceph-mon[41734]: pgmap v1436: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:03.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:35:03 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:35:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:04 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:35:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:04 smithi090 ceph-mon[33646]: pgmap v1437: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:35:05.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:04 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:35:05.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:04 smithi006 ceph-mon[30108]: pgmap v1437: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:05.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:35:05.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:04 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:35:05.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:04 smithi006 ceph-mon[41734]: pgmap v1437: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:05.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:35:05.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:35:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:35:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:35:07.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:06 smithi090 ceph-mon[33646]: pgmap v1438: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:35:07.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:06 smithi006 ceph-mon[30108]: pgmap v1438: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:35:07.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:06 smithi006 ceph-mon[41734]: pgmap v1438: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:35:09.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:08 smithi090 ceph-mon[33646]: pgmap v1439: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:09.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:08 smithi006 ceph-mon[30108]: pgmap v1439: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:09.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:08 smithi006 ceph-mon[41734]: pgmap v1439: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:11.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:10 smithi090 ceph-mon[33646]: pgmap v1440: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:11.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:10 smithi006 ceph-mon[30108]: pgmap v1440: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:11.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:10 smithi006 ceph-mon[41734]: pgmap v1440: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:13.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:12 smithi090 ceph-mon[33646]: pgmap v1441: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:13.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:12 smithi006 ceph-mon[30108]: pgmap v1441: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:13.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:12 smithi006 ceph-mon[41734]: pgmap v1441: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:13.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:35:13 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:35:14.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:13 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:35:14.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:13 smithi090 ceph-mon[33646]: pgmap v1442: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:14.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:13 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:35:14.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:13 smithi006 ceph-mon[30108]: pgmap v1442: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:14.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:13 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:35:14.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:13 smithi006 ceph-mon[41734]: pgmap v1442: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:15.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:35:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:35:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:35:16.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:16 smithi090 ceph-mon[33646]: pgmap v1443: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:16.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:16 smithi006 ceph-mon[30108]: pgmap v1443: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:16.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:16 smithi006 ceph-mon[41734]: pgmap v1443: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:18.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:18 smithi090 ceph-mon[33646]: pgmap v1444: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:18.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:18 smithi006 ceph-mon[30108]: pgmap v1444: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:18.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:18 smithi006 ceph-mon[41734]: pgmap v1444: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:19.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:35:19.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:35:19.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:35:19.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:35:19.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:35:19.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:35:21.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:20 smithi090 ceph-mon[33646]: pgmap v1445: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:21.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:20 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:35:21.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:20 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:35:21.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:20 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:35:21.176 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:20 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:35:21.176 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:20 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:35:21.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:20 smithi006 ceph-mon[30108]: pgmap v1445: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:21.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:20 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:35:21.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:20 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:35:21.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:20 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:35:21.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:20 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:35:21.211 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:20 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:35:21.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:20 smithi006 ceph-mon[41734]: pgmap v1445: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:21.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:20 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:35:21.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:20 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:35:21.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:20 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:35:21.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:20 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:35:21.212 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:20 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:35:23.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:22 smithi090 ceph-mon[33646]: pgmap v1446: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:23.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:22 smithi006 ceph-mon[41734]: pgmap v1446: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:23.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:22 smithi006 ceph-mon[30108]: pgmap v1446: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:23.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:35:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:35:25.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:24 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:35:25.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:24 smithi090 ceph-mon[33646]: pgmap v1447: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:25.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:24 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:35:25.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:24 smithi006 ceph-mon[30108]: pgmap v1447: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:25.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:24 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:35:25.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:24 smithi006 ceph-mon[41734]: pgmap v1447: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:25.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:35:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:35:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:35:27.176 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:26 smithi090 ceph-mon[33646]: pgmap v1448: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:27.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:26 smithi006 ceph-mon[30108]: pgmap v1448: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:27.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:26 smithi006 ceph-mon[41734]: pgmap v1448: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:29.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:28 smithi090 ceph-mon[33646]: pgmap v1449: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:29.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:28 smithi006 ceph-mon[30108]: pgmap v1449: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:29.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:28 smithi006 ceph-mon[41734]: pgmap v1449: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:31.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:30 smithi090 ceph-mon[33646]: pgmap v1450: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:31.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:30 smithi006 ceph-mon[30108]: pgmap v1450: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:31.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:30 smithi006 ceph-mon[41734]: pgmap v1450: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:33.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:32 smithi090 ceph-mon[33646]: pgmap v1451: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:33.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:32 smithi006 ceph-mon[30108]: pgmap v1451: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:33.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:32 smithi006 ceph-mon[41734]: pgmap v1451: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:33.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:35:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:35:35.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:34 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:35:35.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:34 smithi090 ceph-mon[33646]: pgmap v1452: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:35.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:35:35.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:34 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:35:35.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:34 smithi006 ceph-mon[30108]: pgmap v1452: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:35.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:35:35.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:34 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:35:35.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:34 smithi006 ceph-mon[41734]: pgmap v1452: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:35.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:35:35.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:35:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:35:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:35:37.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:36 smithi090 ceph-mon[33646]: pgmap v1453: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:37.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:36 smithi006 ceph-mon[41734]: pgmap v1453: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:37.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:36 smithi006 ceph-mon[30108]: pgmap v1453: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:39.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:38 smithi090 ceph-mon[33646]: pgmap v1454: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:39.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:38 smithi006 ceph-mon[30108]: pgmap v1454: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:39.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:38 smithi006 ceph-mon[41734]: pgmap v1454: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:41.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:40 smithi090 ceph-mon[33646]: pgmap v1455: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:41.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:40 smithi006 ceph-mon[30108]: pgmap v1455: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:41.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:40 smithi006 ceph-mon[41734]: pgmap v1455: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:43.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:42 smithi090 ceph-mon[33646]: pgmap v1456: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:43.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:42 smithi006 ceph-mon[30108]: pgmap v1456: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:43.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:42 smithi006 ceph-mon[41734]: pgmap v1456: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:43.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:35:43 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:35:45.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:44 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:35:45.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:44 smithi090 ceph-mon[33646]: pgmap v1457: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:45.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:44 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:35:45.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:44 smithi006 ceph-mon[30108]: pgmap v1457: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:45.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:44 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:35:45.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:44 smithi006 ceph-mon[41734]: pgmap v1457: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:45.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:35:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:35:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:35:47.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:46 smithi090 ceph-mon[33646]: pgmap v1458: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:35:47.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:46 smithi006 ceph-mon[30108]: pgmap v1458: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:35:47.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:46 smithi006 ceph-mon[41734]: pgmap v1458: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:35:49.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:48 smithi090 ceph-mon[33646]: pgmap v1459: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:49.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:48 smithi006 ceph-mon[30108]: pgmap v1459: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:49.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:48 smithi006 ceph-mon[41734]: pgmap v1459: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:50.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:49 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:35:50.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:49 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:35:50.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:49 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:35:51.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:50 smithi090 ceph-mon[33646]: pgmap v1460: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:35:51.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:50 smithi006 ceph-mon[30108]: pgmap v1460: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:35:51.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:50 smithi006 ceph-mon[41734]: pgmap v1460: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:35:52.425 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:52 smithi090 ceph-mon[33646]: pgmap v1461: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:52.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:52 smithi006 ceph-mon[30108]: pgmap v1461: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:52.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:52 smithi006 ceph-mon[41734]: pgmap v1461: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:35:53.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:35:53 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:35:54.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:54 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:35:54.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:54 smithi090 ceph-mon[33646]: pgmap v1462: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:54.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:54 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:35:54.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:54 smithi006 ceph-mon[30108]: pgmap v1462: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:54.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:54 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:35:54.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:54 smithi006 ceph-mon[41734]: pgmap v1462: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:55.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:35:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:35:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:35:56.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:56 smithi090 ceph-mon[33646]: pgmap v1463: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:35:56.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:56 smithi006 ceph-mon[30108]: pgmap v1463: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:35:56.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:56 smithi006 ceph-mon[41734]: pgmap v1463: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:35:58.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:35:58 smithi090 ceph-mon[33646]: pgmap v1464: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:58.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:35:58 smithi006 ceph-mon[30108]: pgmap v1464: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:35:58.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:35:58 smithi006 ceph-mon[41734]: pgmap v1464: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:00.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:00 smithi090 ceph-mon[33646]: pgmap v1465: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:36:00.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:00 smithi006 ceph-mon[30108]: pgmap v1465: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:36:00.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:00 smithi006 ceph-mon[41734]: pgmap v1465: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:36:02.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:02 smithi090 ceph-mon[33646]: pgmap v1466: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:02.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:02 smithi006 ceph-mon[30108]: pgmap v1466: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:02.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:02 smithi006 ceph-mon[41734]: pgmap v1466: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:03.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:36:03 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:36:04.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:04 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:36:04.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:04 smithi090 ceph-mon[33646]: pgmap v1467: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:04.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:36:04.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:04 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:36:04.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:04 smithi006 ceph-mon[30108]: pgmap v1467: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:04.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:36:04.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:04 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:36:04.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:04 smithi006 ceph-mon[41734]: pgmap v1467: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:04.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:36:05.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:36:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:36:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:36:06.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:06 smithi090 ceph-mon[33646]: pgmap v1468: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:06.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:06 smithi006 ceph-mon[30108]: pgmap v1468: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:06.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:06 smithi006 ceph-mon[41734]: pgmap v1468: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:08.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:08 smithi090 ceph-mon[33646]: pgmap v1469: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:08.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:08 smithi006 ceph-mon[30108]: pgmap v1469: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:08.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:08 smithi006 ceph-mon[41734]: pgmap v1469: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:10.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:10 smithi090 ceph-mon[33646]: pgmap v1470: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:10.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:10 smithi006 ceph-mon[30108]: pgmap v1470: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:10.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:10 smithi006 ceph-mon[41734]: pgmap v1470: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:12.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:12 smithi090 ceph-mon[33646]: pgmap v1471: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:12.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:12 smithi006 ceph-mon[30108]: pgmap v1471: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:12.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:12 smithi006 ceph-mon[41734]: pgmap v1471: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:13.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:36:13 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:36:14.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:14 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:36:14.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:14 smithi090 ceph-mon[33646]: pgmap v1472: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:14.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:14 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:36:14.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:14 smithi006 ceph-mon[30108]: pgmap v1472: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:14.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:14 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:36:14.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:14 smithi006 ceph-mon[41734]: pgmap v1472: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:15.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:36:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:36:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:36:16.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:16 smithi090 ceph-mon[33646]: pgmap v1473: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:36:16.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:16 smithi006 ceph-mon[30108]: pgmap v1473: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:36:16.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:16 smithi006 ceph-mon[41734]: pgmap v1473: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:36:18.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:18 smithi090 ceph-mon[33646]: pgmap v1474: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:18.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:18 smithi006 ceph-mon[30108]: pgmap v1474: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:18.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:18 smithi006 ceph-mon[41734]: pgmap v1474: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:19.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:36:19.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:36:19.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:36:20.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:20 smithi090 ceph-mon[33646]: pgmap v1475: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:36:20.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:20 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:36:20.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:20 smithi006 ceph-mon[30108]: pgmap v1475: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:36:20.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:20 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:36:20.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:20 smithi006 ceph-mon[41734]: pgmap v1475: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:36:20.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:20 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:36:21.662 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.663 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.663 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.663 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.663 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.663 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.663 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.663 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.663 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.663 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.663 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.664 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.664 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.664 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.664 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.664 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.664 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.664 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.664 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.664 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.664 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.664 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.664 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.664 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.665 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.665 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.665 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.665 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.665 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.665 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.665 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.665 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.665 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.665 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.665 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.665 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.665 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.665 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.666 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.666 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.666 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.666 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.666 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.666 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.666 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.666 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.666 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.666 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.666 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.666 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.667 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.667 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.667 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.667 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.667 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.667 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.667 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.667 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.667 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.667 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.667 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.667 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.667 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.668 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.668 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.668 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.668 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.668 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.668 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.668 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.668 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.668 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.668 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.668 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.668 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.668 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.669 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.669 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.669 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.669 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.669 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.669 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.669 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.669 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.669 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.669 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.669 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.669 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.670 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.670 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.670 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.670 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.670 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.670 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.670 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.670 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.670 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.670 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.670 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:21.670 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:36:22.425 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:22 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:36:22.425 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:22 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:36:22.425 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:22 smithi090 ceph-mon[33646]: pgmap v1476: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:22.426 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:22 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:36:22.426 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:22 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:36:22.426 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:22 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:36:22.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:22 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:36:22.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:22 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:36:22.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:22 smithi006 ceph-mon[30108]: pgmap v1476: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:22.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:22 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:36:22.459 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:22 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:36:22.459 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:22 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:36:22.459 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:22 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:36:22.459 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:22 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:36:22.459 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:22 smithi006 ceph-mon[41734]: pgmap v1476: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:22.459 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:22 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:36:22.459 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:22 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:36:22.459 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:22 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:36:23.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:36:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:36:24.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:24 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:36:24.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:24 smithi090 ceph-mon[33646]: pgmap v1477: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:24.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:24 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:36:24.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:24 smithi006 ceph-mon[30108]: pgmap v1477: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:24.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:24 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:36:24.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:24 smithi006 ceph-mon[41734]: pgmap v1477: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:25.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:36:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:36:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:36:26.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:26 smithi090 ceph-mon[33646]: pgmap v1478: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:36:26.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:26 smithi006 ceph-mon[30108]: pgmap v1478: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:36:26.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:26 smithi006 ceph-mon[41734]: pgmap v1478: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:36:28.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:28 smithi090 ceph-mon[33646]: pgmap v1479: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:28.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:28 smithi006 ceph-mon[30108]: pgmap v1479: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:28.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:28 smithi006 ceph-mon[41734]: pgmap v1479: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:30.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:30 smithi090 ceph-mon[33646]: pgmap v1480: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:30.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:30 smithi006 ceph-mon[30108]: pgmap v1480: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:30.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:30 smithi006 ceph-mon[41734]: pgmap v1480: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:32.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:32 smithi090 ceph-mon[33646]: pgmap v1481: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:32.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:32 smithi006 ceph-mon[30108]: pgmap v1481: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:32.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:32 smithi006 ceph-mon[41734]: pgmap v1481: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:33.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:36:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:36:34.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:34 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:36:34.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:34 smithi090 ceph-mon[33646]: pgmap v1482: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:34.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:36:34.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:34 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:36:34.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:34 smithi006 ceph-mon[30108]: pgmap v1482: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:34.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:36:34.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:34 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:36:34.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:34 smithi006 ceph-mon[41734]: pgmap v1482: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:34.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:36:35.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:36:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:36:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:36:36.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:36 smithi090 ceph-mon[33646]: pgmap v1483: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:36.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:36 smithi006 ceph-mon[30108]: pgmap v1483: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:36.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:36 smithi006 ceph-mon[41734]: pgmap v1483: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:38.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:38 smithi090 ceph-mon[33646]: pgmap v1484: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:38.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:38 smithi006 ceph-mon[30108]: pgmap v1484: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:38.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:38 smithi006 ceph-mon[41734]: pgmap v1484: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:40.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:40 smithi090 ceph-mon[33646]: pgmap v1485: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:36:40.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:40 smithi006 ceph-mon[30108]: pgmap v1485: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:36:40.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:40 smithi006 ceph-mon[41734]: pgmap v1485: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:36:42.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:42 smithi090 ceph-mon[33646]: pgmap v1486: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:42.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:42 smithi006 ceph-mon[30108]: pgmap v1486: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:42.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:42 smithi006 ceph-mon[41734]: pgmap v1486: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:43.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:36:43 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:36:44.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:44 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:36:44.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:44 smithi090 ceph-mon[33646]: pgmap v1487: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:44.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:44 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:36:44.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:44 smithi006 ceph-mon[30108]: pgmap v1487: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:44.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:44 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:36:44.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:44 smithi006 ceph-mon[41734]: pgmap v1487: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:45.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:36:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:36:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:36:46.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:46 smithi090 ceph-mon[33646]: pgmap v1488: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:36:46.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:46 smithi006 ceph-mon[30108]: pgmap v1488: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:36:46.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:46 smithi006 ceph-mon[41734]: pgmap v1488: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:36:48.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:48 smithi090 ceph-mon[33646]: pgmap v1489: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:48.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:48 smithi006 ceph-mon[30108]: pgmap v1489: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:48.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:48 smithi006 ceph-mon[41734]: pgmap v1489: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:49.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:49 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:36:49.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:49 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:36:49.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:49 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:36:50.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:50 smithi090 ceph-mon[33646]: pgmap v1490: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:50.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:50 smithi006 ceph-mon[30108]: pgmap v1490: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:50.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:50 smithi006 ceph-mon[41734]: pgmap v1490: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:52.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:52 smithi090 ceph-mon[33646]: pgmap v1491: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:52.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:52 smithi006 ceph-mon[30108]: pgmap v1491: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:52.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:52 smithi006 ceph-mon[41734]: pgmap v1491: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:53.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:36:53 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:36:54.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:54 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:36:54.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:54 smithi090 ceph-mon[33646]: pgmap v1492: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:54.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:54 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:36:54.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:54 smithi006 ceph-mon[30108]: pgmap v1492: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:54.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:54 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:36:54.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:54 smithi006 ceph-mon[41734]: pgmap v1492: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:55.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:36:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:36:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:36:56.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:56 smithi090 ceph-mon[33646]: pgmap v1493: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:56.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:56 smithi006 ceph-mon[30108]: pgmap v1493: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:56.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:56 smithi006 ceph-mon[41734]: pgmap v1493: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:36:58.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:36:58 smithi090 ceph-mon[33646]: pgmap v1494: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:58.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:36:58 smithi006 ceph-mon[30108]: pgmap v1494: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:36:58.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:36:58 smithi006 ceph-mon[41734]: pgmap v1494: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:00.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:00 smithi090 ceph-mon[33646]: pgmap v1495: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:37:00.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:00 smithi006 ceph-mon[30108]: pgmap v1495: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:37:00.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:00 smithi006 ceph-mon[41734]: pgmap v1495: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:37:02.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:02 smithi090 ceph-mon[33646]: pgmap v1496: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:02.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:02 smithi006 ceph-mon[30108]: pgmap v1496: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:02.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:02 smithi006 ceph-mon[41734]: pgmap v1496: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:03.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:37:03 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:37:04.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:04 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:37:04.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:04 smithi090 ceph-mon[33646]: pgmap v1497: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:04.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:37:04.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:04 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:37:04.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:04 smithi006 ceph-mon[30108]: pgmap v1497: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:04.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:37:04.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:04 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:37:04.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:04 smithi006 ceph-mon[41734]: pgmap v1497: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:04.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:37:05.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:37:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:37:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:37:06.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:06 smithi090 ceph-mon[33646]: pgmap v1498: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:37:06.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:06 smithi006 ceph-mon[30108]: pgmap v1498: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:37:06.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:06 smithi006 ceph-mon[41734]: pgmap v1498: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:37:08.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:08 smithi090 ceph-mon[33646]: pgmap v1499: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:08.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:08 smithi006 ceph-mon[30108]: pgmap v1499: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:08.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:08 smithi006 ceph-mon[41734]: pgmap v1499: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:10.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:10 smithi090 ceph-mon[33646]: pgmap v1500: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:37:10.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:10 smithi006 ceph-mon[30108]: pgmap v1500: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:37:10.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:10 smithi006 ceph-mon[41734]: pgmap v1500: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:37:12.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:12 smithi006 ceph-mon[30108]: pgmap v1501: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:12.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:12 smithi006 ceph-mon[41734]: pgmap v1501: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:13.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:12 smithi090 ceph-mon[33646]: pgmap v1501: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:13.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:37:13 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:37:14.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:14 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:37:14.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:14 smithi006 ceph-mon[30108]: pgmap v1502: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:14.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:14 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:37:14.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:14 smithi006 ceph-mon[41734]: pgmap v1502: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:15.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:14 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:37:15.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:14 smithi090 ceph-mon[33646]: pgmap v1502: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:15.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:37:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:37:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:37:16.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:16 smithi006 ceph-mon[30108]: pgmap v1503: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:37:16.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:16 smithi006 ceph-mon[41734]: pgmap v1503: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:37:17.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:16 smithi090 ceph-mon[33646]: pgmap v1503: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:37:18.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:18 smithi006 ceph-mon[30108]: pgmap v1504: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:18.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:18 smithi006 ceph-mon[41734]: pgmap v1504: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:19.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:18 smithi090 ceph-mon[33646]: pgmap v1504: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:19.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:37:19.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:37:20.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:37:20.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:20 smithi006 ceph-mon[30108]: pgmap v1505: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:37:20.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:20 smithi006 ceph-mon[41734]: pgmap v1505: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:37:21.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:20 smithi090 ceph-mon[33646]: pgmap v1505: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:37:22.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:22 smithi006 ceph-mon[30108]: pgmap v1506: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:22.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:22 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:37:22.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:22 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:37:22.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:22 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:37:22.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:22 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:37:22.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:22 smithi006 ceph-mon[41734]: pgmap v1506: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:22.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:22 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:37:22.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:22 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:37:22.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:22 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:37:22.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:22 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:37:23.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:22 smithi090 ceph-mon[33646]: pgmap v1506: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:23.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:22 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:37:23.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:22 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:37:23.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:22 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:37:23.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:22 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:37:23.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:37:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:37:24.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:24 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:37:24.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:24 smithi006 ceph-mon[30108]: pgmap v1507: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:24.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:24 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:37:24.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:24 smithi006 ceph-mon[41734]: pgmap v1507: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:25.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:24 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:37:25.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:24 smithi090 ceph-mon[33646]: pgmap v1507: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:25.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:37:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:37:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:37:26.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:26 smithi006 ceph-mon[30108]: pgmap v1508: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:26.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:26 smithi006 ceph-mon[41734]: pgmap v1508: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:26.977 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:26 smithi090 ceph-mon[33646]: pgmap v1508: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:28.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:28 smithi006 ceph-mon[30108]: pgmap v1509: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:28.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:28 smithi006 ceph-mon[41734]: pgmap v1509: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:29.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:28 smithi090 ceph-mon[33646]: pgmap v1509: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:30.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:30 smithi006 ceph-mon[30108]: pgmap v1510: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:30.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:30 smithi006 ceph-mon[41734]: pgmap v1510: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:31.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:30 smithi090 ceph-mon[33646]: pgmap v1510: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:33.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:32 smithi090 ceph-mon[33646]: pgmap v1511: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:33.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:32 smithi006 ceph-mon[30108]: pgmap v1511: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:33.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:32 smithi006 ceph-mon[41734]: pgmap v1511: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:33.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:37:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:37:35.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:34 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:37:35.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:34 smithi090 ceph-mon[33646]: pgmap v1512: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:35.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:37:35.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:34 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:37:35.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:34 smithi006 ceph-mon[30108]: pgmap v1512: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:35.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:37:35.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:34 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:37:35.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:34 smithi006 ceph-mon[41734]: pgmap v1512: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:35.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:37:35.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:37:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:37:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:37:37.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:36 smithi090 ceph-mon[33646]: pgmap v1513: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:37:37.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:36 smithi006 ceph-mon[30108]: pgmap v1513: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:37:37.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:36 smithi006 ceph-mon[41734]: pgmap v1513: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:37:39.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:38 smithi090 ceph-mon[33646]: pgmap v1514: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:39.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:38 smithi006 ceph-mon[30108]: pgmap v1514: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:39.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:38 smithi006 ceph-mon[41734]: pgmap v1514: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:41.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:40 smithi090 ceph-mon[33646]: pgmap v1515: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:41.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:40 smithi006 ceph-mon[30108]: pgmap v1515: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:41.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:40 smithi006 ceph-mon[41734]: pgmap v1515: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:43.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:42 smithi090 ceph-mon[33646]: pgmap v1516: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:43.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:42 smithi006 ceph-mon[30108]: pgmap v1516: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:43.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:42 smithi006 ceph-mon[41734]: pgmap v1516: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:43.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:37:43 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:37:45.009 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:44 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:37:45.009 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:44 smithi006 ceph-mon[30108]: pgmap v1517: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:45.010 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:44 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:37:45.010 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:44 smithi006 ceph-mon[41734]: pgmap v1517: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:45.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:44 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:37:45.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:44 smithi090 ceph-mon[33646]: pgmap v1517: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:45.458 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:37:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:37:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:37:47.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:46 smithi090 ceph-mon[33646]: pgmap v1518: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:47.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:46 smithi006 ceph-mon[30108]: pgmap v1518: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:47.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:46 smithi006 ceph-mon[41734]: pgmap v1518: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:49.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:48 smithi090 ceph-mon[33646]: pgmap v1519: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:49.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:48 smithi006 ceph-mon[30108]: pgmap v1519: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:49.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:48 smithi006 ceph-mon[41734]: pgmap v1519: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:50.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:49 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:37:50.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:49 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:37:50.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:49 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:37:51.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:50 smithi090 ceph-mon[33646]: pgmap v1520: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:51.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:50 smithi006 ceph-mon[30108]: pgmap v1520: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:51.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:50 smithi006 ceph-mon[41734]: pgmap v1520: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:53.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:52 smithi090 ceph-mon[33646]: pgmap v1521: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:53.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:52 smithi006 ceph-mon[30108]: pgmap v1521: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:53.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:52 smithi006 ceph-mon[41734]: pgmap v1521: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:37:53.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:37:53 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:37:55.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:54 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:37:55.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:54 smithi090 ceph-mon[33646]: pgmap v1522: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:55.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:54 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:37:55.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:54 smithi006 ceph-mon[30108]: pgmap v1522: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:55.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:54 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:37:55.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:54 smithi006 ceph-mon[41734]: pgmap v1522: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:55.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:37:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:37:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:37:57.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:56 smithi090 ceph-mon[33646]: pgmap v1523: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:37:57.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:56 smithi006 ceph-mon[30108]: pgmap v1523: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:37:57.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:56 smithi006 ceph-mon[41734]: pgmap v1523: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:37:59.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:37:58 smithi090 ceph-mon[33646]: pgmap v1524: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:59.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:37:58 smithi006 ceph-mon[30108]: pgmap v1524: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:37:59.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:37:58 smithi006 ceph-mon[41734]: pgmap v1524: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:01.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:00 smithi090 ceph-mon[33646]: pgmap v1525: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:01.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:00 smithi006 ceph-mon[30108]: pgmap v1525: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:01.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:00 smithi006 ceph-mon[41734]: pgmap v1525: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:01.674 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.674 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.674 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.675 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.675 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.675 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.675 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.675 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.675 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.675 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.675 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.675 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.675 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.675 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.675 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.675 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.676 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.676 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.676 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.676 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.676 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.676 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.676 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.676 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.676 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.676 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.676 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.676 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.676 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.676 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.676 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.677 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.677 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.677 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.677 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.677 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.677 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.677 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.677 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.677 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.677 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.677 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.677 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.677 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.677 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.678 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.678 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.678 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.678 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.678 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.678 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.678 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.678 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.678 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.678 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.678 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.678 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.678 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.678 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.679 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.679 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.679 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.679 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.679 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.679 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.679 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.679 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.679 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.679 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.679 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.679 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.679 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.679 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.679 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.680 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.680 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.680 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.680 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.680 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.680 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.680 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.680 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.680 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.680 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.680 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.680 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.680 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.680 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.681 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.681 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.681 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.681 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.681 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.681 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.681 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.681 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.681 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.681 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.681 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:01.681 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:38:03.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:02 smithi090 ceph-mon[33646]: pgmap v1526: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:03.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:02 smithi006 ceph-mon[30108]: pgmap v1526: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:03.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:02 smithi006 ceph-mon[41734]: pgmap v1526: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:03.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:38:03 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:38:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:04 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:38:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:04 smithi090 ceph-mon[33646]: pgmap v1527: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:38:05.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:04 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:38:05.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:04 smithi006 ceph-mon[30108]: pgmap v1527: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:05.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:38:05.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:04 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:38:05.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:04 smithi006 ceph-mon[41734]: pgmap v1527: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:05.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:38:05.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:38:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:38:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:38:07.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:06 smithi090 ceph-mon[33646]: pgmap v1528: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:07.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:06 smithi006 ceph-mon[30108]: pgmap v1528: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:07.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:06 smithi006 ceph-mon[41734]: pgmap v1528: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:09.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:08 smithi090 ceph-mon[33646]: pgmap v1529: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:09.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:08 smithi006 ceph-mon[30108]: pgmap v1529: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:09.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:08 smithi006 ceph-mon[41734]: pgmap v1529: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:11.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:10 smithi090 ceph-mon[33646]: pgmap v1530: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:11.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:10 smithi006 ceph-mon[30108]: pgmap v1530: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:11.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:10 smithi006 ceph-mon[41734]: pgmap v1530: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:13.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:12 smithi090 ceph-mon[33646]: pgmap v1531: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:13.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:12 smithi006 ceph-mon[30108]: pgmap v1531: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:13.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:12 smithi006 ceph-mon[41734]: pgmap v1531: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:13.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:38:13 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:38:15.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:14 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:38:15.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:14 smithi090 ceph-mon[33646]: pgmap v1532: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:15.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:14 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:38:15.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:14 smithi006 ceph-mon[30108]: pgmap v1532: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:15.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:14 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:38:15.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:14 smithi006 ceph-mon[41734]: pgmap v1532: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:15.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:38:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:38:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:38:17.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:16 smithi090 ceph-mon[33646]: pgmap v1533: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:17.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:16 smithi006 ceph-mon[30108]: pgmap v1533: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:17.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:16 smithi006 ceph-mon[41734]: pgmap v1533: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:19.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:18 smithi090 ceph-mon[33646]: pgmap v1534: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:19.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:18 smithi006 ceph-mon[30108]: pgmap v1534: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:19.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:18 smithi006 ceph-mon[41734]: pgmap v1534: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:20.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:38:20.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:38:20.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:38:21.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:20 smithi090 ceph-mon[33646]: pgmap v1535: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:21.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:20 smithi006 ceph-mon[30108]: pgmap v1535: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:21.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:20 smithi006 ceph-mon[41734]: pgmap v1535: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:23.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:22 smithi090 ceph-mon[33646]: pgmap v1536: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:23.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:22 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:38:23.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:22 smithi006 ceph-mon[30108]: pgmap v1536: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:23.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:22 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:38:23.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:22 smithi006 ceph-mon[41734]: pgmap v1536: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:23.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:22 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:38:23.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:38:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:38:25.159 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:24 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:38:25.159 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:24 smithi006 ceph-mon[30108]: pgmap v1537: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:25.159 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:24 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:38:25.159 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:24 smithi006 ceph-mon[41734]: pgmap v1537: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:25.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:24 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:38:25.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:24 smithi090 ceph-mon[33646]: pgmap v1537: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:25.457 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:38:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:38:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:38:26.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:26 smithi006 ceph-mon[30108]: pgmap v1538: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:26.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:26 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:38:26.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:26 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:38:26.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:26 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:38:26.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:26 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:38:26.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:26 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:38:26.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:26 smithi006 ceph-mon[41734]: pgmap v1538: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:26.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:26 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:38:26.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:26 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:38:26.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:26 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:38:26.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:26 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:38:26.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:26 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:38:27.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:26 smithi090 ceph-mon[33646]: pgmap v1538: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:27.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:26 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:38:27.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:26 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:38:27.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:26 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:38:27.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:26 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:38:27.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:26 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:38:29.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:28 smithi090 ceph-mon[33646]: pgmap v1539: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:29.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:28 smithi006 ceph-mon[30108]: pgmap v1539: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:29.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:28 smithi006 ceph-mon[41734]: pgmap v1539: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:31.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:30 smithi090 ceph-mon[33646]: pgmap v1540: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:31.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:30 smithi006 ceph-mon[30108]: pgmap v1540: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:31.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:30 smithi006 ceph-mon[41734]: pgmap v1540: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:33.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:32 smithi090 ceph-mon[33646]: pgmap v1541: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:33.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:32 smithi006 ceph-mon[30108]: pgmap v1541: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:33.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:32 smithi006 ceph-mon[41734]: pgmap v1541: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:33.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:38:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:38:35.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:34 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:38:35.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:34 smithi090 ceph-mon[33646]: pgmap v1542: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:35.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:38:35.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:34 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:38:35.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:34 smithi006 ceph-mon[30108]: pgmap v1542: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:35.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:38:35.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:34 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:38:35.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:34 smithi006 ceph-mon[41734]: pgmap v1542: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:35.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:38:35.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:38:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:38:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:38:37.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:36 smithi090 ceph-mon[33646]: pgmap v1543: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:37.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:36 smithi006 ceph-mon[30108]: pgmap v1543: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:37.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:36 smithi006 ceph-mon[41734]: pgmap v1543: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:39.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:38 smithi090 ceph-mon[33646]: pgmap v1544: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:39.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:38 smithi006 ceph-mon[30108]: pgmap v1544: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:39.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:38 smithi006 ceph-mon[41734]: pgmap v1544: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:41.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:40 smithi090 ceph-mon[33646]: pgmap v1545: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:41.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:40 smithi006 ceph-mon[30108]: pgmap v1545: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:41.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:40 smithi006 ceph-mon[41734]: pgmap v1545: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:43.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:42 smithi090 ceph-mon[33646]: pgmap v1546: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:43.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:42 smithi006 ceph-mon[30108]: pgmap v1546: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:43.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:42 smithi006 ceph-mon[41734]: pgmap v1546: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:43.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:38:43 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:38:45.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:44 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:38:45.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:44 smithi090 ceph-mon[33646]: pgmap v1547: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:45.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:44 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:38:45.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:44 smithi006 ceph-mon[30108]: pgmap v1547: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:45.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:44 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:38:45.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:44 smithi006 ceph-mon[41734]: pgmap v1547: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:45.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:38:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:38:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:38:47.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:46 smithi090 ceph-mon[33646]: pgmap v1548: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:47.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:46 smithi006 ceph-mon[30108]: pgmap v1548: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:47.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:46 smithi006 ceph-mon[41734]: pgmap v1548: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:49.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:48 smithi090 ceph-mon[33646]: pgmap v1549: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:49.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:48 smithi006 ceph-mon[30108]: pgmap v1549: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:49.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:48 smithi006 ceph-mon[41734]: pgmap v1549: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:50.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:49 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:38:50.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:49 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:38:50.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:49 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:38:51.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:50 smithi090 ceph-mon[33646]: pgmap v1550: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:51.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:50 smithi006 ceph-mon[30108]: pgmap v1550: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:51.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:50 smithi006 ceph-mon[41734]: pgmap v1550: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:52.425 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:52 smithi090 ceph-mon[33646]: pgmap v1551: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:52.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:52 smithi006 ceph-mon[30108]: pgmap v1551: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:52.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:52 smithi006 ceph-mon[41734]: pgmap v1551: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:38:53.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:38:53 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:38:54.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:54 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:38:54.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:54 smithi090 ceph-mon[33646]: pgmap v1552: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:54.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:54 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:38:54.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:54 smithi006 ceph-mon[30108]: pgmap v1552: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:54.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:54 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:38:54.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:54 smithi006 ceph-mon[41734]: pgmap v1552: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:55.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:38:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:38:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:38:56.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:56 smithi090 ceph-mon[33646]: pgmap v1553: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:56.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:56 smithi006 ceph-mon[30108]: pgmap v1553: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:56.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:56 smithi006 ceph-mon[41734]: pgmap v1553: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:38:58.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:38:58 smithi090 ceph-mon[33646]: pgmap v1554: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:58.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:38:58 smithi006 ceph-mon[30108]: pgmap v1554: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:38:58.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:38:58 smithi006 ceph-mon[41734]: pgmap v1554: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:00.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:00 smithi090 ceph-mon[33646]: pgmap v1555: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:00.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:00 smithi006 ceph-mon[30108]: pgmap v1555: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:00.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:00 smithi006 ceph-mon[41734]: pgmap v1555: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:02.425 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:02 smithi090 ceph-mon[33646]: pgmap v1556: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:02.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:02 smithi006 ceph-mon[30108]: pgmap v1556: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:02.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:02 smithi006 ceph-mon[41734]: pgmap v1556: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:03.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:39:03 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:39:04.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:04 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:39:04.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:04 smithi090 ceph-mon[33646]: pgmap v1557: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:04.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:39:04.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:04 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:39:04.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:04 smithi006 ceph-mon[30108]: pgmap v1557: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:04.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:39:04.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:04 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:39:04.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:04 smithi006 ceph-mon[41734]: pgmap v1557: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:04.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:39:05.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:39:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:39:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:39:06.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:06 smithi090 ceph-mon[33646]: pgmap v1558: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:39:06.957 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:06 smithi006 ceph-mon[41734]: pgmap v1558: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:39:06.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:06 smithi006 ceph-mon[30108]: pgmap v1558: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:39:08.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:08 smithi090 ceph-mon[33646]: pgmap v1559: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:08.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:08 smithi006 ceph-mon[30108]: pgmap v1559: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:08.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:08 smithi006 ceph-mon[41734]: pgmap v1559: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:10.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:10 smithi090 ceph-mon[33646]: pgmap v1560: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:39:10.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:10 smithi006 ceph-mon[30108]: pgmap v1560: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:39:10.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:10 smithi006 ceph-mon[41734]: pgmap v1560: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:39:12.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:12 smithi090 ceph-mon[33646]: pgmap v1561: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:12.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:12 smithi006 ceph-mon[30108]: pgmap v1561: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:12.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:12 smithi006 ceph-mon[41734]: pgmap v1561: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:13.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:39:13 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:39:14.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:14 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:39:14.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:14 smithi090 ceph-mon[33646]: pgmap v1562: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:14.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:14 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:39:14.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:14 smithi006 ceph-mon[30108]: pgmap v1562: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:14.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:14 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:39:14.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:14 smithi006 ceph-mon[41734]: pgmap v1562: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:15.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:39:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:39:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:39:16.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:16 smithi006 ceph-mon[30108]: pgmap v1563: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:16.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:16 smithi006 ceph-mon[41734]: pgmap v1563: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:17.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:16 smithi090 ceph-mon[33646]: pgmap v1563: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:18.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:18 smithi006 ceph-mon[30108]: pgmap v1564: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:18.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:18 smithi006 ceph-mon[41734]: pgmap v1564: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:19.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:18 smithi090 ceph-mon[33646]: pgmap v1564: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:19.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:39:19.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:39:20.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:39:20.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:20 smithi006 ceph-mon[30108]: pgmap v1565: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:20.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:20 smithi006 ceph-mon[41734]: pgmap v1565: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:21.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:20 smithi090 ceph-mon[33646]: pgmap v1565: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:22.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:22 smithi006 ceph-mon[30108]: pgmap v1566: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:22.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:22 smithi006 ceph-mon[41734]: pgmap v1566: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:23.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:22 smithi090 ceph-mon[33646]: pgmap v1566: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:23.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:39:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:39:24.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:24 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:39:24.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:24 smithi006 ceph-mon[30108]: pgmap v1567: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:24.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:24 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:39:24.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:24 smithi006 ceph-mon[41734]: pgmap v1567: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:25.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:24 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:39:25.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:24 smithi090 ceph-mon[33646]: pgmap v1567: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:25.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:39:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:39:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:39:26.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:26 smithi006 ceph-mon[30108]: pgmap v1568: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:26.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:26 smithi006 ceph-mon[41734]: pgmap v1568: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:26.976 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:26 smithi090 ceph-mon[33646]: pgmap v1568: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:27.985 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:27 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:39:28.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:27 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:39:28.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:27 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:39:29.117 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:28 smithi090 ceph-mon[33646]: pgmap v1569: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:29.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:28 smithi006 ceph-mon[30108]: pgmap v1569: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:29.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:28 smithi006 ceph-mon[41734]: pgmap v1569: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:30.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:30 smithi090 ceph-mon[33646]: pgmap v1570: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:31.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:30 smithi006 ceph-mon[30108]: pgmap v1570: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:31.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:30 smithi006 ceph-mon[41734]: pgmap v1570: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:32.425 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:32 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:39:32.425 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:32 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:39:32.425 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:32 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:39:32.425 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:32 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:39:32.425 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:32 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:39:32.425 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:32 smithi090 ceph-mon[33646]: pgmap v1571: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:32.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:32 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:39:32.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:32 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:39:32.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:32 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:39:32.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:32 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:39:32.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:32 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:39:32.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:32 smithi006 ceph-mon[30108]: pgmap v1571: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:32.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:32 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:39:32.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:32 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:39:32.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:32 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:39:32.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:32 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:39:32.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:32 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:39:32.459 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:32 smithi006 ceph-mon[41734]: pgmap v1571: 228 pgs: 228 active+clean; 582 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:33.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:39:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:39:34.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:34 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:39:34.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:34 smithi090 ceph-mon[33646]: pgmap v1572: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:39:34.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:39:34.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:34 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:39:34.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:34 smithi006 ceph-mon[30108]: pgmap v1572: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:39:34.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:39:34.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:34 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:39:34.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:34 smithi006 ceph-mon[41734]: pgmap v1572: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:39:34.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:39:35.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:39:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:39:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:39:36.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:36 smithi090 ceph-mon[33646]: pgmap v1573: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:39:36.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:36 smithi006 ceph-mon[30108]: pgmap v1573: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:39:36.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:36 smithi006 ceph-mon[41734]: pgmap v1573: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:39:38.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:38 smithi006 ceph-mon[30108]: pgmap v1574: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:39:38.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:38 smithi006 ceph-mon[41734]: pgmap v1574: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:39:39.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:38 smithi090 ceph-mon[33646]: pgmap v1574: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:39:40.686 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.686 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.686 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.686 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.686 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.686 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.687 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.687 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.687 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.687 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.687 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.687 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.687 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.687 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.687 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.687 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.687 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.687 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.687 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.688 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.688 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.688 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.688 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.688 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.688 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.688 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.688 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.688 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.688 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.688 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.688 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.688 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.688 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.689 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.689 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.689 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.689 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.689 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.689 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.689 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.689 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.689 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.689 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.689 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.689 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.689 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.689 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.689 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.690 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.690 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.690 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.690 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.690 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.690 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.690 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.690 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.690 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.690 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.690 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.690 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.690 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.691 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.691 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.691 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.691 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.691 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.691 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.691 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.691 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.691 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.691 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.691 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.691 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.691 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.691 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.692 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.692 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.692 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.692 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.692 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.692 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.692 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.692 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.692 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.692 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.692 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.692 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.692 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.693 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.693 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.693 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.693 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.693 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.693 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.693 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.693 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.693 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.693 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.693 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:39:40.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:40 smithi006 ceph-mon[30108]: pgmap v1575: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:39:40.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:40 smithi006 ceph-mon[41734]: pgmap v1575: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:39:41.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:40 smithi090 ceph-mon[33646]: pgmap v1575: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:39:42.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:42 smithi006 ceph-mon[30108]: pgmap v1576: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:39:42.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:42 smithi006 ceph-mon[41734]: pgmap v1576: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:39:43.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:42 smithi090 ceph-mon[33646]: pgmap v1576: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:39:43.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:39:43 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:39:44.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:44 smithi006 ceph-mon[30108]: pgmap v1577: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:39:44.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:44 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:39:44.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:44 smithi006 ceph-mon[41734]: pgmap v1577: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:39:44.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:44 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:39:45.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:44 smithi090 ceph-mon[33646]: pgmap v1577: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-24T12:39:45.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:44 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:39:45.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:39:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:39:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:39:46.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:46 smithi006 ceph-mon[30108]: pgmap v1578: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:46.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:46 smithi006 ceph-mon[41734]: pgmap v1578: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:47.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:46 smithi090 ceph-mon[33646]: pgmap v1578: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:48.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:48 smithi006 ceph-mon[30108]: pgmap v1579: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:48.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:48 smithi006 ceph-mon[41734]: pgmap v1579: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:49.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:48 smithi090 ceph-mon[33646]: pgmap v1579: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:49.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:49 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:39:49.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:49 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:39:50.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:49 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:39:50.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:50 smithi006 ceph-mon[30108]: pgmap v1580: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:50.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:50 smithi006 ceph-mon[41734]: pgmap v1580: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:51.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:50 smithi090 ceph-mon[33646]: pgmap v1580: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:52.425 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:52 smithi090 ceph-mon[33646]: pgmap v1581: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:52.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:52 smithi006 ceph-mon[30108]: pgmap v1581: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:52.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:52 smithi006 ceph-mon[41734]: pgmap v1581: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:53.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:39:53 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:39:54.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:54 smithi090 ceph-mon[33646]: pgmap v1582: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:54.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:54 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:39:54.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:54 smithi006 ceph-mon[30108]: pgmap v1582: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:54.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:54 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:39:54.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:54 smithi006 ceph-mon[41734]: pgmap v1582: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:54.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:54 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:39:55.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:39:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:39:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:39:56.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:56 smithi006 ceph-mon[30108]: pgmap v1583: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:56.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:56 smithi006 ceph-mon[41734]: pgmap v1583: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:56.998 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:56 smithi090 ceph-mon[33646]: pgmap v1583: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:39:58.957 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:39:58 smithi006 ceph-mon[41734]: pgmap v1584: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:58.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:39:58 smithi006 ceph-mon[30108]: pgmap v1584: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:39:59.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:39:58 smithi090 ceph-mon[33646]: pgmap v1584: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:00.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:00 smithi006 ceph-mon[30108]: pgmap v1585: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:00.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:00 smithi006 ceph-mon[30108]: overall HEALTH_WARN 3 pool(s) do not have an application enabled 2024-03-24T12:40:00.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:00 smithi006 ceph-mon[41734]: pgmap v1585: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:00.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:00 smithi006 ceph-mon[41734]: overall HEALTH_WARN 3 pool(s) do not have an application enabled 2024-03-24T12:40:01.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:00 smithi090 ceph-mon[33646]: pgmap v1585: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:01.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:00 smithi090 ceph-mon[33646]: overall HEALTH_WARN 3 pool(s) do not have an application enabled 2024-03-24T12:40:02.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:02 smithi006 ceph-mon[30108]: pgmap v1586: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:02.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:02 smithi006 ceph-mon[41734]: pgmap v1586: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:03.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:02 smithi090 ceph-mon[33646]: pgmap v1586: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:03.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:40:03 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:40:04.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:04 smithi006 ceph-mon[30108]: pgmap v1587: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:04.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:04 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:40:04.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:40:04.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:04 smithi006 ceph-mon[41734]: pgmap v1587: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:04 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:40:04.961 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:40:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:04 smithi090 ceph-mon[33646]: pgmap v1587: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:04 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:40:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:40:05.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:40:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:40:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:40:06.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:06 smithi006 ceph-mon[30108]: pgmap v1588: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:06.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:06 smithi006 ceph-mon[41734]: pgmap v1588: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:07.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:06 smithi090 ceph-mon[33646]: pgmap v1588: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:08.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:08 smithi006 ceph-mon[30108]: pgmap v1589: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:08.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:08 smithi006 ceph-mon[41734]: pgmap v1589: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:09.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:08 smithi090 ceph-mon[33646]: pgmap v1589: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:10.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:10 smithi006 ceph-mon[30108]: pgmap v1590: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:10.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:10 smithi006 ceph-mon[41734]: pgmap v1590: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:11.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:10 smithi090 ceph-mon[33646]: pgmap v1590: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:12.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:12 smithi006 ceph-mon[30108]: pgmap v1591: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:12.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:12 smithi006 ceph-mon[41734]: pgmap v1591: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:13.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:12 smithi090 ceph-mon[33646]: pgmap v1591: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:13.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:40:13 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:40:14.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:14 smithi006 ceph-mon[41734]: pgmap v1592: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:14.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:14 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:40:14.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:14 smithi006 ceph-mon[30108]: pgmap v1592: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:14.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:14 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:40:15.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:14 smithi090 ceph-mon[33646]: pgmap v1592: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:15.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:14 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:40:15.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:40:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:40:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:40:16.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:16 smithi006 ceph-mon[41734]: pgmap v1593: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:40:16.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:16 smithi006 ceph-mon[30108]: pgmap v1593: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:40:17.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:16 smithi090 ceph-mon[33646]: pgmap v1593: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:40:19.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:18 smithi090 ceph-mon[33646]: pgmap v1594: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:19.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:18 smithi006 ceph-mon[30108]: pgmap v1594: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:19.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:18 smithi006 ceph-mon[41734]: pgmap v1594: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:20.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:40:20.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:40:20.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:40:21.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:20 smithi090 ceph-mon[33646]: pgmap v1595: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:40:21.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:20 smithi006 ceph-mon[30108]: pgmap v1595: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:40:21.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:20 smithi006 ceph-mon[41734]: pgmap v1595: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:40:23.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:22 smithi090 ceph-mon[33646]: pgmap v1596: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:23.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:22 smithi006 ceph-mon[30108]: pgmap v1596: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:23.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:22 smithi006 ceph-mon[41734]: pgmap v1596: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:23.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:40:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:40:25.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:24 smithi090 ceph-mon[33646]: pgmap v1597: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:25.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:24 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:40:25.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:24 smithi006 ceph-mon[30108]: pgmap v1597: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:25.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:24 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:40:25.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:24 smithi006 ceph-mon[41734]: pgmap v1597: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:25.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:24 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:40:25.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:40:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:40:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:40:26.978 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:26 smithi090 ceph-mon[33646]: pgmap v1598: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:40:27.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:26 smithi006 ceph-mon[30108]: pgmap v1598: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:40:27.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:26 smithi006 ceph-mon[41734]: pgmap v1598: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:40:29.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:28 smithi090 ceph-mon[33646]: pgmap v1599: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:29.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:28 smithi006 ceph-mon[30108]: pgmap v1599: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:29.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:28 smithi006 ceph-mon[41734]: pgmap v1599: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:31.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:30 smithi090 ceph-mon[33646]: pgmap v1600: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:31.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:30 smithi006 ceph-mon[30108]: pgmap v1600: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:31.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:30 smithi006 ceph-mon[41734]: pgmap v1600: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:32.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:31 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:40:32.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:31 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:40:32.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:31 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:40:33.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:32 smithi090 ceph-mon[33646]: pgmap v1601: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:33.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:32 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:40:33.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:32 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:40:33.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:32 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:40:33.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:32 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:40:33.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:32 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:40:33.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:32 smithi006 ceph-mon[30108]: pgmap v1601: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:33.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:32 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:40:33.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:32 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:40:33.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:32 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:40:33.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:32 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:40:33.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:32 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:40:33.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:32 smithi006 ceph-mon[41734]: pgmap v1601: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:33.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:32 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:40:33.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:32 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:40:33.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:32 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:40:33.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:32 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:40:33.209 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:32 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:40:33.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:40:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:40:35.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:34 smithi090 ceph-mon[33646]: pgmap v1602: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:35.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:34 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:40:35.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:40:35.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:34 smithi006 ceph-mon[30108]: pgmap v1602: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:35.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:34 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:40:35.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:40:35.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:34 smithi006 ceph-mon[41734]: pgmap v1602: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:35.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:34 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:40:35.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:40:35.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:40:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:40:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:40:37.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:36 smithi090 ceph-mon[33646]: pgmap v1603: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:37.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:36 smithi006 ceph-mon[30108]: pgmap v1603: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:37.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:36 smithi006 ceph-mon[41734]: pgmap v1603: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:39.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:38 smithi090 ceph-mon[33646]: pgmap v1604: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:39.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:38 smithi006 ceph-mon[30108]: pgmap v1604: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:39.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:38 smithi006 ceph-mon[41734]: pgmap v1604: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:41.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:40 smithi090 ceph-mon[33646]: pgmap v1605: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:41.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:40 smithi006 ceph-mon[30108]: pgmap v1605: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:41.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:40 smithi006 ceph-mon[41734]: pgmap v1605: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:43.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:42 smithi090 ceph-mon[33646]: pgmap v1606: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:43.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:42 smithi006 ceph-mon[30108]: pgmap v1606: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:43.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:42 smithi006 ceph-mon[41734]: pgmap v1606: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:43.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:40:43 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:40:45.009 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:44 smithi006 ceph-mon[30108]: pgmap v1607: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:45.009 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:44 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:40:45.009 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:44 smithi006 ceph-mon[41734]: pgmap v1607: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:45.009 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:44 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:40:45.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:44 smithi090 ceph-mon[33646]: pgmap v1607: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:45.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:44 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:40:45.457 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:40:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:40:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:40:47.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:46 smithi090 ceph-mon[33646]: pgmap v1608: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:40:47.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:46 smithi006 ceph-mon[30108]: pgmap v1608: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:40:47.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:46 smithi006 ceph-mon[41734]: pgmap v1608: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:40:49.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:48 smithi090 ceph-mon[33646]: pgmap v1609: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:49.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:48 smithi006 ceph-mon[30108]: pgmap v1609: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:49.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:48 smithi006 ceph-mon[41734]: pgmap v1609: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:50.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:49 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:40:50.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:49 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:40:50.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:49 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:40:51.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:50 smithi090 ceph-mon[33646]: pgmap v1610: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:40:51.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:50 smithi006 ceph-mon[30108]: pgmap v1610: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:40:51.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:50 smithi006 ceph-mon[41734]: pgmap v1610: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:40:53.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:52 smithi090 ceph-mon[33646]: pgmap v1611: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:53.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:52 smithi006 ceph-mon[30108]: pgmap v1611: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:53.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:52 smithi006 ceph-mon[41734]: pgmap v1611: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:40:54.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:40:53 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:40:55.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:54 smithi090 ceph-mon[33646]: pgmap v1612: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:55.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:54 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:40:55.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:54 smithi006 ceph-mon[30108]: pgmap v1612: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:55.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:54 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:40:55.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:54 smithi006 ceph-mon[41734]: pgmap v1612: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:55.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:54 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:40:55.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:40:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:40:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:40:57.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:56 smithi090 ceph-mon[33646]: pgmap v1613: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:40:57.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:56 smithi006 ceph-mon[30108]: pgmap v1613: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:40:57.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:56 smithi006 ceph-mon[41734]: pgmap v1613: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:40:59.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:40:58 smithi090 ceph-mon[33646]: pgmap v1614: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:59.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:40:58 smithi006 ceph-mon[30108]: pgmap v1614: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:40:59.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:40:58 smithi006 ceph-mon[41734]: pgmap v1614: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:01.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:00 smithi090 ceph-mon[33646]: pgmap v1615: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:01.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:00 smithi006 ceph-mon[30108]: pgmap v1615: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:01.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:00 smithi006 ceph-mon[41734]: pgmap v1615: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:03.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:02 smithi090 ceph-mon[33646]: pgmap v1616: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:03.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:02 smithi006 ceph-mon[30108]: pgmap v1616: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:03.211 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:02 smithi006 ceph-mon[41734]: pgmap v1616: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:04.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:41:03 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:41:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:04 smithi090 ceph-mon[33646]: pgmap v1617: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:04 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:41:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:41:05.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:04 smithi006 ceph-mon[30108]: pgmap v1617: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:05.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:04 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:41:05.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:41:05.209 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:04 smithi006 ceph-mon[41734]: pgmap v1617: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:05.209 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:04 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:41:05.209 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:41:05.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:41:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:41:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:41:07.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:06 smithi090 ceph-mon[33646]: pgmap v1618: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:07.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:06 smithi006 ceph-mon[30108]: pgmap v1618: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:07.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:06 smithi006 ceph-mon[41734]: pgmap v1618: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:09.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:08 smithi090 ceph-mon[33646]: pgmap v1619: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:09.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:08 smithi006 ceph-mon[30108]: pgmap v1619: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:09.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:08 smithi006 ceph-mon[41734]: pgmap v1619: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:10.175 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:41:09 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a[66673]: logger=cleanup t=2024-03-24T12:41:09.790741325Z level=info msg="Completed cleanup jobs" duration=4.674312ms 2024-03-24T12:41:11.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:10 smithi090 ceph-mon[33646]: pgmap v1620: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:11.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:10 smithi006 ceph-mon[30108]: pgmap v1620: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:11.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:10 smithi006 ceph-mon[41734]: pgmap v1620: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:13.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:12 smithi090 ceph-mon[33646]: pgmap v1621: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:13.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:12 smithi006 ceph-mon[30108]: pgmap v1621: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:13.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:12 smithi006 ceph-mon[41734]: pgmap v1621: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:14.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:41:13 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:41:15.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:14 smithi090 ceph-mon[33646]: pgmap v1622: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:15.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:14 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:41:15.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:14 smithi006 ceph-mon[30108]: pgmap v1622: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:15.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:14 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:41:15.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:14 smithi006 ceph-mon[41734]: pgmap v1622: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:15.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:14 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:41:15.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:41:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:41:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:41:17.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:16 smithi090 ceph-mon[33646]: pgmap v1623: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:17.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:16 smithi006 ceph-mon[30108]: pgmap v1623: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:17.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:16 smithi006 ceph-mon[41734]: pgmap v1623: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:19.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:18 smithi090 ceph-mon[33646]: pgmap v1624: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:19.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:18 smithi006 ceph-mon[30108]: pgmap v1624: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:19.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:18 smithi006 ceph-mon[41734]: pgmap v1624: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:20.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:41:20.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:41:20.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:41:20.697 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.698 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.698 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.698 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.698 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.698 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.698 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.698 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.698 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.698 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.698 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.698 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.699 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.699 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.699 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.699 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.699 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.699 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.699 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.699 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.699 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.699 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.699 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.699 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.699 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.699 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.700 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.700 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.700 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.700 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.700 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.700 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.700 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.700 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.700 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.700 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.700 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.700 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.700 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.700 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.700 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.701 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.701 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.701 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.701 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.701 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.701 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.701 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.701 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.701 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.701 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.701 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.701 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.701 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.701 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.701 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.702 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.702 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.702 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.702 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.702 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.702 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.702 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.702 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.702 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.702 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.702 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.702 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.702 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.702 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.703 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.703 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.703 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.703 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.703 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.703 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.703 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.703 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.703 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.703 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.703 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.704 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.704 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.704 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.704 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.704 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.704 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.704 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.704 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.704 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.704 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.704 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.704 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.704 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.705 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.705 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.705 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.705 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.705 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:20.705 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:41:21.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:20 smithi090 ceph-mon[33646]: pgmap v1625: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:21.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:20 smithi006 ceph-mon[30108]: pgmap v1625: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:21.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:20 smithi006 ceph-mon[41734]: pgmap v1625: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:23.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:22 smithi090 ceph-mon[33646]: pgmap v1626: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:23.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:22 smithi006 ceph-mon[30108]: pgmap v1626: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:23.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:22 smithi006 ceph-mon[41734]: pgmap v1626: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:24.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:41:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:41:25.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:24 smithi090 ceph-mon[33646]: pgmap v1627: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:25.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:24 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:41:25.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:24 smithi006 ceph-mon[30108]: pgmap v1627: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:25.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:24 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:41:25.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:24 smithi006 ceph-mon[41734]: pgmap v1627: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:25.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:24 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:41:25.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:41:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:41:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:41:27.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:26 smithi090 ceph-mon[33646]: pgmap v1628: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:27.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:26 smithi006 ceph-mon[30108]: pgmap v1628: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:27.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:26 smithi006 ceph-mon[41734]: pgmap v1628: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:29.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:28 smithi090 ceph-mon[33646]: pgmap v1629: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:29.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:28 smithi006 ceph-mon[30108]: pgmap v1629: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:29.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:28 smithi006 ceph-mon[41734]: pgmap v1629: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:31.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:30 smithi090 ceph-mon[33646]: pgmap v1630: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:31.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:30 smithi006 ceph-mon[30108]: pgmap v1630: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:31.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:30 smithi006 ceph-mon[41734]: pgmap v1630: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:33.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:32 smithi090 ceph-mon[33646]: pgmap v1631: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:33.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:32 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:41:33.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:32 smithi006 ceph-mon[30108]: pgmap v1631: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:33.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:32 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:41:33.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:32 smithi006 ceph-mon[41734]: pgmap v1631: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:33.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:32 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:41:34.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:41:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:41:34.924 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:34 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:41:34.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:34 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:41:34.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:41:34.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:41:34.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:34 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:41:34.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:34 smithi090 ceph-mon[33646]: pgmap v1632: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:34.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:34 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:41:34.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:34 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:41:34.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:34 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:41:34.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:41:34.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:41:34.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:34 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:41:34.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:34 smithi006 ceph-mon[30108]: pgmap v1632: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:34.959 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:34 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:41:34.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:34 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:41:34.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:34 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:41:34.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:41:34.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:41:34.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:34 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:41:34.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:34 smithi006 ceph-mon[41734]: pgmap v1632: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:34.959 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:34 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:41:35.524 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:41:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:41:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:41:35.924 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:35 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:41:35.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:35 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:41:35.957 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:35 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:41:36.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:36 smithi090 ceph-mon[33646]: pgmap v1633: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:36.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:36 smithi006 ceph-mon[30108]: pgmap v1633: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:36.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:36 smithi006 ceph-mon[41734]: pgmap v1633: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:39.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:38 smithi090 ceph-mon[33646]: pgmap v1634: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:39.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:38 smithi006 ceph-mon[30108]: pgmap v1634: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:39.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:38 smithi006 ceph-mon[41734]: pgmap v1634: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:41.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:40 smithi090 ceph-mon[33646]: pgmap v1635: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:41.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:40 smithi006 ceph-mon[30108]: pgmap v1635: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:41.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:40 smithi006 ceph-mon[41734]: pgmap v1635: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:43.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:42 smithi090 ceph-mon[33646]: pgmap v1636: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:43.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:42 smithi006 ceph-mon[30108]: pgmap v1636: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:43.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:42 smithi006 ceph-mon[41734]: pgmap v1636: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:44.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:41:43 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:41:45.008 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:44 smithi006 ceph-mon[30108]: pgmap v1637: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:45.008 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:44 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:41:45.009 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:44 smithi006 ceph-mon[41734]: pgmap v1637: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:45.009 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:44 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:41:45.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:44 smithi090 ceph-mon[33646]: pgmap v1637: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:45.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:44 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:41:45.457 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:41:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:41:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:41:47.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:46 smithi090 ceph-mon[33646]: pgmap v1638: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:47.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:46 smithi006 ceph-mon[30108]: pgmap v1638: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:47.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:46 smithi006 ceph-mon[41734]: pgmap v1638: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:49.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:48 smithi090 ceph-mon[33646]: pgmap v1639: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:49.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:48 smithi006 ceph-mon[30108]: pgmap v1639: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:49.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:48 smithi006 ceph-mon[41734]: pgmap v1639: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:50.178 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:49 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:41:50.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:49 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:41:50.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:49 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:41:51.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:50 smithi090 ceph-mon[33646]: pgmap v1640: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:51.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:50 smithi006 ceph-mon[30108]: pgmap v1640: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:51.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:50 smithi006 ceph-mon[41734]: pgmap v1640: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:41:53.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:52 smithi090 ceph-mon[33646]: pgmap v1641: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:53.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:52 smithi006 ceph-mon[30108]: pgmap v1641: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:53.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:52 smithi006 ceph-mon[41734]: pgmap v1641: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:54.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:41:53 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:41:55.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:54 smithi090 ceph-mon[33646]: pgmap v1642: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:55.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:54 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:41:55.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:54 smithi006 ceph-mon[30108]: pgmap v1642: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:55.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:54 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:41:55.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:54 smithi006 ceph-mon[41734]: pgmap v1642: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:55.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:54 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:41:55.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:41:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:41:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:41:57.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:56 smithi090 ceph-mon[33646]: pgmap v1643: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:57.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:56 smithi006 ceph-mon[30108]: pgmap v1643: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:57.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:56 smithi006 ceph-mon[41734]: pgmap v1643: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:41:59.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:41:58 smithi090 ceph-mon[33646]: pgmap v1644: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:59.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:41:58 smithi006 ceph-mon[30108]: pgmap v1644: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:41:59.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:41:58 smithi006 ceph-mon[41734]: pgmap v1644: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:42:01.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:00 smithi090 ceph-mon[33646]: pgmap v1645: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:42:01.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:00 smithi006 ceph-mon[30108]: pgmap v1645: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:42:01.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:00 smithi006 ceph-mon[41734]: pgmap v1645: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:42:03.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:02 smithi090 ceph-mon[33646]: pgmap v1646: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:42:03.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:02 smithi006 ceph-mon[30108]: pgmap v1646: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:42:03.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:02 smithi006 ceph-mon[41734]: pgmap v1646: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:42:04.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:42:03 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:42:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:04 smithi090 ceph-mon[33646]: pgmap v1647: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:42:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:04 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:42:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:42:05.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:04 smithi006 ceph-mon[30108]: pgmap v1647: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:42:05.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:04 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:42:05.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:42:05.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:04 smithi006 ceph-mon[41734]: pgmap v1647: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:42:05.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:04 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:42:05.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:42:05.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:42:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:42:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:42:07.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:06 smithi090 ceph-mon[33646]: pgmap v1648: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:42:07.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:06 smithi006 ceph-mon[30108]: pgmap v1648: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:42:07.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:06 smithi006 ceph-mon[41734]: pgmap v1648: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:42:09.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:08 smithi090 ceph-mon[33646]: pgmap v1649: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:42:09.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:08 smithi006 ceph-mon[30108]: pgmap v1649: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:42:09.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:08 smithi006 ceph-mon[41734]: pgmap v1649: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:42:11.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:10 smithi090 ceph-mon[33646]: pgmap v1650: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:42:11.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:10 smithi006 ceph-mon[30108]: pgmap v1650: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:42:11.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:10 smithi006 ceph-mon[41734]: pgmap v1650: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:42:13.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:12 smithi090 ceph-mon[33646]: pgmap v1651: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:42:13.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:12 smithi006 ceph-mon[30108]: pgmap v1651: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:42:13.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:12 smithi006 ceph-mon[41734]: pgmap v1651: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:42:13.704 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.705 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.705 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.705 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.705 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.705 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.705 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.705 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.705 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.705 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.705 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.705 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.705 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.705 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.705 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.706 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.706 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.706 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.706 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.706 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.706 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.706 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.706 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.706 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.706 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.706 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.706 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.706 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.707 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.707 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.707 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.707 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.707 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.707 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.707 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.707 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.707 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.707 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.707 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.707 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.708 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.708 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.708 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.708 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.708 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.708 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.708 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.708 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.708 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.708 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.709 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.709 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.709 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 1 sec 2024-03-24T12:42:13.709 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: started 512 aios 2024-03-24T12:42:13.709 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4aa65600 2024-03-24T12:42:13.709 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada2900 2024-03-24T12:42:13.709 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada3250 2024-03-24T12:42:13.709 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada2db0 2024-03-24T12:42:13.709 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada3730 2024-03-24T12:42:13.709 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada41c0 2024-03-24T12:42:13.709 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada3fc0 2024-03-24T12:42:13.709 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada4690 2024-03-24T12:42:13.709 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada4b00 2024-03-24T12:42:13.709 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada4c10 2024-03-24T12:42:13.710 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada50d0 2024-03-24T12:42:13.710 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada5270 2024-03-24T12:42:13.710 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada47a0 2024-03-24T12:42:13.710 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada48b0 2024-03-24T12:42:13.710 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada49c0 2024-03-24T12:42:13.710 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada5380 2024-03-24T12:42:13.710 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada4d20 2024-03-24T12:42:13.710 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada4e30 2024-03-24T12:42:13.710 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada5840 2024-03-24T12:42:13.710 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada5d00 2024-03-24T12:42:13.710 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada5950 2024-03-24T12:42:13.710 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada5a80 2024-03-24T12:42:13.711 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada5e10 2024-03-24T12:42:13.711 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada5f20 2024-03-24T12:42:13.711 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada5bf0 2024-03-24T12:42:13.711 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada6390 2024-03-24T12:42:13.711 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada6030 2024-03-24T12:42:13.711 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada6140 2024-03-24T12:42:13.711 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada6250 2024-03-24T12:42:13.711 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada6970 2024-03-24T12:42:13.711 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada64d0 2024-03-24T12:42:13.711 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada6600 2024-03-24T12:42:13.711 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada6730 2024-03-24T12:42:13.711 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada6860 2024-03-24T12:42:13.711 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada6f40 2024-03-24T12:42:13.711 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada6aa0 2024-03-24T12:42:13.712 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada6bf0 2024-03-24T12:42:13.712 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada6d00 2024-03-24T12:42:13.712 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada6e10 2024-03-24T12:42:13.712 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada7530 2024-03-24T12:42:13.712 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada7050 2024-03-24T12:42:13.712 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada7160 2024-03-24T12:42:13.712 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada7270 2024-03-24T12:42:13.712 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada73a0 2024-03-24T12:42:13.712 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada7b40 2024-03-24T12:42:13.712 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada7660 2024-03-24T12:42:13.712 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada77c0 2024-03-24T12:42:13.712 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada78f0 2024-03-24T12:42:13.712 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada7c50 2024-03-24T12:42:13.713 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada80e0 2024-03-24T12:42:13.713 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada8550 2024-03-24T12:42:13.713 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada8680 2024-03-24T12:42:13.713 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada7d60 2024-03-24T12:42:13.713 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada7eb0 2024-03-24T12:42:13.713 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada5490 2024-03-24T12:42:13.713 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada55c0 2024-03-24T12:42:13.713 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada5710 2024-03-24T12:42:13.713 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada7fc0 2024-03-24T12:42:13.713 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada8c50 2024-03-24T12:42:13.713 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada87b0 2024-03-24T12:42:13.713 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada88c0 2024-03-24T12:42:13.713 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada8aa0 2024-03-24T12:42:13.713 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada8d80 2024-03-24T12:42:13.714 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada8ed0 2024-03-24T12:42:13.714 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada9480 2024-03-24T12:42:13.714 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada8fe0 2024-03-24T12:42:13.714 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada9110 2024-03-24T12:42:13.714 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada9220 2024-03-24T12:42:13.714 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada9330 2024-03-24T12:42:13.714 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada9a30 2024-03-24T12:42:13.714 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada95e0 2024-03-24T12:42:13.714 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada9730 2024-03-24T12:42:13.714 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada9860 2024-03-24T12:42:13.714 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada9b60 2024-03-24T12:42:13.714 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada9c70 2024-03-24T12:42:13.714 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaa240 2024-03-24T12:42:13.714 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada9da0 2024-03-24T12:42:13.715 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada9ed0 2024-03-24T12:42:13.715 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaa020 2024-03-24T12:42:13.715 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaa350 2024-03-24T12:42:13.715 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaa7c0 2024-03-24T12:42:13.715 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaa8d0 2024-03-24T12:42:13.715 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada81f0 2024-03-24T12:42:13.715 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada8300 2024-03-24T12:42:13.715 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4ada8430 2024-03-24T12:42:13.715 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaaa20 2024-03-24T12:42:13.715 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaae90 2024-03-24T12:42:13.715 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaab30 2024-03-24T12:42:13.715 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaac40 2024-03-24T12:42:13.715 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaad50 2024-03-24T12:42:13.715 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adab480 2024-03-24T12:42:13.715 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaafe0 2024-03-24T12:42:13.716 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adab0f0 2024-03-24T12:42:13.716 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adab200 2024-03-24T12:42:13.716 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adab350 2024-03-24T12:42:13.716 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaba30 2024-03-24T12:42:13.716 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adab590 2024-03-24T12:42:13.716 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adab6a0 2024-03-24T12:42:13.716 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adab7d0 2024-03-24T12:42:13.716 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adab900 2024-03-24T12:42:13.716 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adac020 2024-03-24T12:42:13.716 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adabb80 2024-03-24T12:42:13.717 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adabc90 2024-03-24T12:42:13.717 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adabda0 2024-03-24T12:42:13.717 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adabed0 2024-03-24T12:42:13.717 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adac5f0 2024-03-24T12:42:13.717 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adac150 2024-03-24T12:42:13.717 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adac260 2024-03-24T12:42:13.717 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adac370 2024-03-24T12:42:13.717 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adac4a0 2024-03-24T12:42:13.718 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adacbc0 2024-03-24T12:42:13.718 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adac720 2024-03-24T12:42:13.718 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adac830 2024-03-24T12:42:13.718 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adac960 2024-03-24T12:42:13.718 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaca90 2024-03-24T12:42:13.718 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adad170 2024-03-24T12:42:13.718 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaccd0 2024-03-24T12:42:13.718 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adacde0 2024-03-24T12:42:13.718 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adacf10 2024-03-24T12:42:13.718 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adad020 2024-03-24T12:42:13.718 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adad760 2024-03-24T12:42:13.718 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adad2c0 2024-03-24T12:42:13.719 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adad3d0 2024-03-24T12:42:13.719 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adad500 2024-03-24T12:42:13.719 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adad630 2024-03-24T12:42:13.719 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adadd60 2024-03-24T12:42:13.719 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adad8c0 2024-03-24T12:42:13.719 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adad9d0 2024-03-24T12:42:13.719 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adadb00 2024-03-24T12:42:13.719 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adade90 2024-03-24T12:42:13.719 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adae340 2024-03-24T12:42:13.719 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adae450 2024-03-24T12:42:13.720 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adadfc0 2024-03-24T12:42:13.720 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adae130 2024-03-24T12:42:13.720 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaea20 2024-03-24T12:42:13.720 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adae580 2024-03-24T12:42:13.720 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adae6b0 2024-03-24T12:42:13.720 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adae800 2024-03-24T12:42:13.720 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adae910 2024-03-24T12:42:13.720 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaeff0 2024-03-24T12:42:13.720 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaeb50 2024-03-24T12:42:13.720 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaec80 2024-03-24T12:42:13.720 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaedb0 2024-03-24T12:42:13.720 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaeee0 2024-03-24T12:42:13.720 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaf5c0 2024-03-24T12:42:13.721 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaf120 2024-03-24T12:42:13.721 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaf250 2024-03-24T12:42:13.721 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaf380 2024-03-24T12:42:13.721 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaf4b0 2024-03-24T12:42:13.721 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adafb90 2024-03-24T12:42:13.721 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaf6f0 2024-03-24T12:42:13.721 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaf840 2024-03-24T12:42:13.721 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaf950 2024-03-24T12:42:13.721 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adafa80 2024-03-24T12:42:13.721 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb0160 2024-03-24T12:42:13.721 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adafcc0 2024-03-24T12:42:13.721 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adafdf0 2024-03-24T12:42:13.721 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaff00 2024-03-24T12:42:13.721 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb0050 2024-03-24T12:42:13.722 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb0730 2024-03-24T12:42:13.722 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb0290 2024-03-24T12:42:13.722 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb03c0 2024-03-24T12:42:13.722 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb04d0 2024-03-24T12:42:13.722 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb0600 2024-03-24T12:42:13.722 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb0d00 2024-03-24T12:42:13.722 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb0860 2024-03-24T12:42:13.722 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb0970 2024-03-24T12:42:13.722 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb0ac0 2024-03-24T12:42:13.722 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb0bf0 2024-03-24T12:42:13.722 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb1330 2024-03-24T12:42:13.722 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb0e90 2024-03-24T12:42:13.723 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb0fc0 2024-03-24T12:42:13.723 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb10d0 2024-03-24T12:42:13.723 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb1220 2024-03-24T12:42:13.723 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb1900 2024-03-24T12:42:13.723 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb1460 2024-03-24T12:42:13.723 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb1590 2024-03-24T12:42:13.723 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb16a0 2024-03-24T12:42:13.723 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb17d0 2024-03-24T12:42:13.723 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb1ed0 2024-03-24T12:42:13.723 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb1a30 2024-03-24T12:42:13.723 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb1b60 2024-03-24T12:42:13.723 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb1c70 2024-03-24T12:42:13.723 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb1dc0 2024-03-24T12:42:13.724 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb24a0 2024-03-24T12:42:13.724 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb2000 2024-03-24T12:42:13.724 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb2130 2024-03-24T12:42:13.724 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb2240 2024-03-24T12:42:13.724 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb2390 2024-03-24T12:42:13.724 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb2a70 2024-03-24T12:42:13.724 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb25d0 2024-03-24T12:42:13.724 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb2700 2024-03-24T12:42:13.724 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb2810 2024-03-24T12:42:13.724 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb2960 2024-03-24T12:42:13.724 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb3080 2024-03-24T12:42:13.724 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb2be0 2024-03-24T12:42:13.724 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb2d10 2024-03-24T12:42:13.724 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb2e20 2024-03-24T12:42:13.725 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb2f70 2024-03-24T12:42:13.725 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb3670 2024-03-24T12:42:13.725 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb31d0 2024-03-24T12:42:13.725 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb3300 2024-03-24T12:42:13.725 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb3430 2024-03-24T12:42:13.725 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb3560 2024-03-24T12:42:13.725 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb3c60 2024-03-24T12:42:13.725 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb37c0 2024-03-24T12:42:13.725 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb38f0 2024-03-24T12:42:13.725 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb3a00 2024-03-24T12:42:13.725 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb3b50 2024-03-24T12:42:13.725 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb4250 2024-03-24T12:42:13.725 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb3db0 2024-03-24T12:42:13.725 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb3ee0 2024-03-24T12:42:13.725 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb3ff0 2024-03-24T12:42:13.726 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb4140 2024-03-24T12:42:13.726 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb4840 2024-03-24T12:42:13.726 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb43a0 2024-03-24T12:42:13.726 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb44d0 2024-03-24T12:42:13.726 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb4600 2024-03-24T12:42:13.726 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb4730 2024-03-24T12:42:13.726 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb4e10 2024-03-24T12:42:13.726 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb4970 2024-03-24T12:42:13.726 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb4ac0 2024-03-24T12:42:13.726 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb4bd0 2024-03-24T12:42:13.726 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb4d00 2024-03-24T12:42:13.726 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb53e0 2024-03-24T12:42:13.726 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb4f40 2024-03-24T12:42:13.726 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb5070 2024-03-24T12:42:13.726 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb5180 2024-03-24T12:42:13.727 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb52b0 2024-03-24T12:42:13.727 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb59d0 2024-03-24T12:42:13.727 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb5530 2024-03-24T12:42:13.727 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb5640 2024-03-24T12:42:13.727 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb57b0 2024-03-24T12:42:13.727 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb58c0 2024-03-24T12:42:13.727 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb5fc0 2024-03-24T12:42:13.727 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb5b20 2024-03-24T12:42:13.727 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb5c50 2024-03-24T12:42:13.727 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb5d60 2024-03-24T12:42:13.727 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb5eb0 2024-03-24T12:42:13.727 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb6590 2024-03-24T12:42:13.727 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb60f0 2024-03-24T12:42:13.727 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb6260 2024-03-24T12:42:13.727 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb6370 2024-03-24T12:42:13.728 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb66a0 2024-03-24T12:42:13.728 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb6b50 2024-03-24T12:42:13.728 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb6c80 2024-03-24T12:42:13.728 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb67d0 2024-03-24T12:42:13.728 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb6920 2024-03-24T12:42:13.728 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb7250 2024-03-24T12:42:13.728 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb6db0 2024-03-24T12:42:13.728 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb6ec0 2024-03-24T12:42:13.728 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb6ff0 2024-03-24T12:42:13.728 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb7120 2024-03-24T12:42:13.728 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb7840 2024-03-24T12:42:13.728 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb73a0 2024-03-24T12:42:13.728 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb74d0 2024-03-24T12:42:13.728 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb75e0 2024-03-24T12:42:13.728 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb7710 2024-03-24T12:42:13.729 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb7e10 2024-03-24T12:42:13.729 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb7970 2024-03-24T12:42:13.729 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb7aa0 2024-03-24T12:42:13.729 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb7bd0 2024-03-24T12:42:13.729 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb7d00 2024-03-24T12:42:13.729 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb83e0 2024-03-24T12:42:13.729 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb7f40 2024-03-24T12:42:13.729 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb8070 2024-03-24T12:42:13.729 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb81c0 2024-03-24T12:42:13.729 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb8510 2024-03-24T12:42:13.729 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb89a0 2024-03-24T12:42:13.729 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb8af0 2024-03-24T12:42:13.729 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb8620 2024-03-24T12:42:13.729 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb8770 2024-03-24T12:42:13.729 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb82d0 2024-03-24T12:42:13.729 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb8c40 2024-03-24T12:42:13.730 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb8d70 2024-03-24T12:42:13.730 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb8880 2024-03-24T12:42:13.730 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb8ee0 2024-03-24T12:42:13.730 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb9390 2024-03-24T12:42:13.730 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb94c0 2024-03-24T12:42:13.730 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb9a90 2024-03-24T12:42:13.730 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb9030 2024-03-24T12:42:13.730 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb9160 2024-03-24T12:42:13.730 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb95f0 2024-03-24T12:42:13.730 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb9720 2024-03-24T12:42:13.730 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb9850 2024-03-24T12:42:13.730 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb9980 2024-03-24T12:42:13.730 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adba060 2024-03-24T12:42:13.730 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb9bc0 2024-03-24T12:42:13.731 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb9cf0 2024-03-24T12:42:13.731 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb9e00 2024-03-24T12:42:13.731 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb9f50 2024-03-24T12:42:13.731 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adb9270 2024-03-24T12:42:13.731 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adba190 2024-03-24T12:42:13.731 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adba670 2024-03-24T12:42:13.731 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adba7a0 2024-03-24T12:42:13.731 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adba2a0 2024-03-24T12:42:13.731 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adba3f0 2024-03-24T12:42:13.731 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adba520 2024-03-24T12:42:13.731 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adba8f0 2024-03-24T12:42:13.731 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbadd0 2024-03-24T12:42:13.731 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbaf70 2024-03-24T12:42:13.731 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbb540 2024-03-24T12:42:13.731 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbaa20 2024-03-24T12:42:13.732 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbab50 2024-03-24T12:42:13.732 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbb0a0 2024-03-24T12:42:13.732 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbb1d0 2024-03-24T12:42:13.732 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbb300 2024-03-24T12:42:13.732 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbb430 2024-03-24T12:42:13.732 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbbb10 2024-03-24T12:42:13.732 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbb670 2024-03-24T12:42:13.732 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbb7a0 2024-03-24T12:42:13.732 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbb8b0 2024-03-24T12:42:13.732 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbb9e0 2024-03-24T12:42:13.732 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbc0e0 2024-03-24T12:42:13.732 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbbc40 2024-03-24T12:42:13.732 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbbd90 2024-03-24T12:42:13.732 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbbea0 2024-03-24T12:42:13.732 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbbfd0 2024-03-24T12:42:13.733 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbc6d0 2024-03-24T12:42:13.733 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbc230 2024-03-24T12:42:13.733 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbc360 2024-03-24T12:42:13.733 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbac80 2024-03-24T12:42:13.733 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbc470 2024-03-24T12:42:13.733 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbc5c0 2024-03-24T12:42:13.733 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbc820 2024-03-24T12:42:13.733 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbccd0 2024-03-24T12:42:13.733 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbce00 2024-03-24T12:42:13.733 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbc950 2024-03-24T12:42:13.733 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbcaa0 2024-03-24T12:42:13.733 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbd3d0 2024-03-24T12:42:13.734 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbcf30 2024-03-24T12:42:13.734 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbd060 2024-03-24T12:42:13.734 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbd190 2024-03-24T12:42:13.734 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbd500 2024-03-24T12:42:13.734 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbda00 2024-03-24T12:42:13.734 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbdb80 2024-03-24T12:42:13.734 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaa460 2024-03-24T12:42:13.734 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adaa590 2024-03-24T12:42:13.734 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbe150 2024-03-24T12:42:13.734 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbdcb0 2024-03-24T12:42:13.734 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbddc0 2024-03-24T12:42:13.734 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbdef0 2024-03-24T12:42:13.734 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbe020 2024-03-24T12:42:13.734 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbe720 2024-03-24T12:42:13.734 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbe280 2024-03-24T12:42:13.735 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbe3d0 2024-03-24T12:42:13.737 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbe500 2024-03-24T12:42:13.737 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbe850 2024-03-24T12:42:13.737 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbed30 2024-03-24T12:42:13.737 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbee60 2024-03-24T12:42:13.737 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbd650 2024-03-24T12:42:13.737 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbd780 2024-03-24T12:42:13.737 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbf410 2024-03-24T12:42:13.737 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbef70 2024-03-24T12:42:13.737 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbf0a0 2024-03-24T12:42:13.737 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbf1d0 2024-03-24T12:42:13.737 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbf300 2024-03-24T12:42:13.738 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbf9e0 2024-03-24T12:42:13.738 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbf540 2024-03-24T12:42:13.738 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbf690 2024-03-24T12:42:13.738 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbf7c0 2024-03-24T12:42:13.738 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbfb10 2024-03-24T12:42:13.738 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbffa0 2024-03-24T12:42:13.738 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc00d0 2024-03-24T12:42:13.738 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbe960 2024-03-24T12:42:13.738 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbea90 2024-03-24T12:42:13.738 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc06a0 2024-03-24T12:42:13.738 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc0200 2024-03-24T12:42:13.738 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc0330 2024-03-24T12:42:13.739 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc0460 2024-03-24T12:42:13.739 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc0590 2024-03-24T12:42:13.739 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc0c70 2024-03-24T12:42:13.739 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc07d0 2024-03-24T12:42:13.739 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc0900 2024-03-24T12:42:13.739 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc0a30 2024-03-24T12:42:13.739 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc0b40 2024-03-24T12:42:13.739 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc1220 2024-03-24T12:42:13.739 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc0d80 2024-03-24T12:42:13.739 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc0eb0 2024-03-24T12:42:13.739 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc0fe0 2024-03-24T12:42:13.739 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc1110 2024-03-24T12:42:13.739 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc17f0 2024-03-24T12:42:13.739 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc1350 2024-03-24T12:42:13.740 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc1480 2024-03-24T12:42:13.740 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc1590 2024-03-24T12:42:13.740 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc16e0 2024-03-24T12:42:13.740 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc1dc0 2024-03-24T12:42:13.740 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc1920 2024-03-24T12:42:13.740 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc1a50 2024-03-24T12:42:13.740 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc1b60 2024-03-24T12:42:13.740 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc1cb0 2024-03-24T12:42:13.740 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc23b0 2024-03-24T12:42:13.740 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc1f10 2024-03-24T12:42:13.740 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc2060 2024-03-24T12:42:13.740 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc2190 2024-03-24T12:42:13.740 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc2500 2024-03-24T12:42:13.740 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc29e0 2024-03-24T12:42:13.741 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc2b60 2024-03-24T12:42:13.741 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc2630 2024-03-24T12:42:13.741 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc2760 2024-03-24T12:42:13.741 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc2890 2024-03-24T12:42:13.741 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc2c90 2024-03-24T12:42:13.741 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc3120 2024-03-24T12:42:13.741 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc3270 2024-03-24T12:42:13.741 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbfc40 2024-03-24T12:42:13.741 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbfd70 2024-03-24T12:42:13.741 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc3840 2024-03-24T12:42:13.741 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc33a0 2024-03-24T12:42:13.741 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc34d0 2024-03-24T12:42:13.741 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc3600 2024-03-24T12:42:13.741 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc3730 2024-03-24T12:42:13.742 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc3e10 2024-03-24T12:42:13.742 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc3970 2024-03-24T12:42:13.742 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc3ac0 2024-03-24T12:42:13.742 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc3bf0 2024-03-24T12:42:13.742 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc3f40 2024-03-24T12:42:13.742 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc43d0 2024-03-24T12:42:13.742 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc4500 2024-03-24T12:42:13.742 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbfe80 2024-03-24T12:42:13.742 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc4680 2024-03-24T12:42:13.742 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc4820 2024-03-24T12:42:13.742 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adbd8b0 2024-03-24T12:42:13.742 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc2da0 2024-03-24T12:42:13.742 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc2ed0 2024-03-24T12:42:13.743 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc4970 2024-03-24T12:42:13.743 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc4e00 2024-03-24T12:42:13.743 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc4f30 2024-03-24T12:42:13.743 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc5500 2024-03-24T12:42:13.743 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc4aa0 2024-03-24T12:42:13.743 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc4bd0 2024-03-24T12:42:13.743 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc5060 2024-03-24T12:42:13.743 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc5190 2024-03-24T12:42:13.743 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc4ce0 2024-03-24T12:42:13.743 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc52c0 2024-03-24T12:42:13.743 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc53f0 2024-03-24T12:42:13.743 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc5650 2024-03-24T12:42:13.743 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc5ac0 2024-03-24T12:42:13.743 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc5c50 2024-03-24T12:42:13.744 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc4050 2024-03-24T12:42:13.744 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc4180 2024-03-24T12:42:13.744 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc6240 2024-03-24T12:42:13.744 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc5da0 2024-03-24T12:42:13.744 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc5ed0 2024-03-24T12:42:13.744 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc6000 2024-03-24T12:42:13.744 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc6130 2024-03-24T12:42:13.744 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc6810 2024-03-24T12:42:13.744 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc6370 2024-03-24T12:42:14.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:42:13 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:42:15.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:14 smithi090 ceph-mon[33646]: pgmap v1652: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:42:15.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:14 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:42:15.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:14 smithi090 ceph-mon[33646]: osdmap e756: 8 total, 8 up, 8 in 2024-03-24T12:42:15.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:14 smithi006 ceph-mon[30108]: pgmap v1652: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:42:15.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:14 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:42:15.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:14 smithi006 ceph-mon[30108]: osdmap e756: 8 total, 8 up, 8 in 2024-03-24T12:42:15.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:14 smithi006 ceph-mon[41734]: pgmap v1652: 228 pgs: 228 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:42:15.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:14 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:42:15.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:14 smithi006 ceph-mon[41734]: osdmap e756: 8 total, 8 up, 8 in 2024-03-24T12:42:15.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:42:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:42:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:42:16.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:15 smithi090 ceph-mon[33646]: osdmap e757: 8 total, 8 up, 8 in 2024-03-24T12:42:16.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:15 smithi006 ceph-mon[30108]: osdmap e757: 8 total, 8 up, 8 in 2024-03-24T12:42:16.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:15 smithi006 ceph-mon[41734]: osdmap e757: 8 total, 8 up, 8 in 2024-03-24T12:42:17.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:16 smithi090 ceph-mon[33646]: pgmap v1655: 164 pgs: 164 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T12:42:17.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:16 smithi090 ceph-mon[33646]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:42:17.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:16 smithi090 ceph-mon[33646]: osdmap e758: 8 total, 8 up, 8 in 2024-03-24T12:42:17.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:16 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:42:17.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:16 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:42:17.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:16 smithi006 ceph-mon[30108]: pgmap v1655: 164 pgs: 164 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T12:42:17.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:16 smithi006 ceph-mon[30108]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:42:17.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:16 smithi006 ceph-mon[30108]: osdmap e758: 8 total, 8 up, 8 in 2024-03-24T12:42:17.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:16 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:42:17.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:16 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:42:17.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:16 smithi006 ceph-mon[41734]: pgmap v1655: 164 pgs: 164 active+clean; 582 KiB data, 879 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-24T12:42:17.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:16 smithi006 ceph-mon[41734]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:42:17.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:16 smithi006 ceph-mon[41734]: osdmap e758: 8 total, 8 up, 8 in 2024-03-24T12:42:17.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:16 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:42:17.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:16 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:42:18.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:42:18.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:17 smithi090 ceph-mon[33646]: osdmap e759: 8 total, 8 up, 8 in 2024-03-24T12:42:18.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-03-24T12:42:18.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:42:18.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:42:18.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:42:18.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:17 smithi006 ceph-mon[30108]: osdmap e759: 8 total, 8 up, 8 in 2024-03-24T12:42:18.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-03-24T12:42:18.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:42:18.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:42:18.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-24T12:42:18.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:17 smithi006 ceph-mon[41734]: osdmap e759: 8 total, 8 up, 8 in 2024-03-24T12:42:18.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-03-24T12:42:18.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:42:18.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:42:19.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:18 smithi090 ceph-mon[33646]: pgmap v1658: 196 pgs: 196 active+clean; 583 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-24T12:42:19.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-24T12:42:19.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:18 smithi090 ceph-mon[33646]: osdmap e760: 8 total, 8 up, 8 in 2024-03-24T12:42:19.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "app2"}]: dispatch 2024-03-24T12:42:19.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:18 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:42:19.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:42:19.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:18 smithi006 ceph-mon[30108]: pgmap v1658: 196 pgs: 196 active+clean; 583 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-24T12:42:19.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-24T12:42:19.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:18 smithi006 ceph-mon[30108]: osdmap e760: 8 total, 8 up, 8 in 2024-03-24T12:42:19.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "app2"}]: dispatch 2024-03-24T12:42:19.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:18 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:42:19.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:42:19.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:18 smithi006 ceph-mon[41734]: pgmap v1658: 196 pgs: 196 active+clean; 583 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-24T12:42:19.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-24T12:42:19.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:18 smithi006 ceph-mon[41734]: osdmap e760: 8 total, 8 up, 8 in 2024-03-24T12:42:19.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "app2"}]: dispatch 2024-03-24T12:42:19.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:18 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:42:19.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-24T12:42:20.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-24T12:42:20.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:19 smithi090 ceph-mon[33646]: osdmap e761: 8 total, 8 up, 8 in 2024-03-24T12:42:20.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi006-81994-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-03-24T12:42:20.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:19 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-24T12:42:20.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:19 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-24T12:42:20.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:42:20.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-24T12:42:20.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:19 smithi006 ceph-mon[30108]: osdmap e761: 8 total, 8 up, 8 in 2024-03-24T12:42:20.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi006-81994-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-03-24T12:42:20.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:19 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-24T12:42:20.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:19 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-24T12:42:20.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:42:20.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi006-81994-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-24T12:42:20.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:19 smithi006 ceph-mon[41734]: osdmap e761: 8 total, 8 up, 8 in 2024-03-24T12:42:20.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi006-81994-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-03-24T12:42:20.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:19 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-24T12:42:20.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:19 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-24T12:42:20.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:42:21.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:20 smithi090 ceph-mon[33646]: pgmap v1661: 196 pgs: 196 active+clean; 583 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-24T12:42:21.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:20 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-24T12:42:21.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:20 smithi090 ceph-mon[33646]: osdmap e762: 8 total, 8 up, 8 in 2024-03-24T12:42:21.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:20 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-24T12:42:21.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:20 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-24T12:42:21.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:20 smithi006 ceph-mon[30108]: pgmap v1661: 196 pgs: 196 active+clean; 583 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-24T12:42:21.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:20 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-24T12:42:21.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:20 smithi006 ceph-mon[30108]: osdmap e762: 8 total, 8 up, 8 in 2024-03-24T12:42:21.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:20 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-24T12:42:21.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:20 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-24T12:42:21.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:20 smithi006 ceph-mon[41734]: pgmap v1661: 196 pgs: 196 active+clean; 583 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-24T12:42:21.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:20 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-24T12:42:21.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:20 smithi006 ceph-mon[41734]: osdmap e762: 8 total, 8 up, 8 in 2024-03-24T12:42:21.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:20 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-24T12:42:21.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:20 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-24T12:42:21.838 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc64c0 2024-03-24T12:42:21.838 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc65f0 2024-03-24T12:42:21.838 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc6940 2024-03-24T12:42:21.838 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc6dd0 2024-03-24T12:42:21.838 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc6f00 2024-03-24T12:42:21.838 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc5760 2024-03-24T12:42:21.838 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc5890 2024-03-24T12:42:21.839 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc74d0 2024-03-24T12:42:21.839 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc7030 2024-03-24T12:42:21.839 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc7160 2024-03-24T12:42:21.839 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc7290 2024-03-24T12:42:21.839 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc73c0 2024-03-24T12:42:21.839 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc7aa0 2024-03-24T12:42:21.839 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc7600 2024-03-24T12:42:21.839 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc7750 2024-03-24T12:42:21.839 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc7880 2024-03-24T12:42:21.839 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc7bd0 2024-03-24T12:42:21.839 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc8060 2024-03-24T12:42:21.839 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc81e0 2024-03-24T12:42:21.839 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc7990 2024-03-24T12:42:21.839 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc8330 2024-03-24T12:42:21.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc87e0 2024-03-24T12:42:21.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc88f0 2024-03-24T12:42:21.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc6a50 2024-03-24T12:42:21.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc6ba0 2024-03-24T12:42:21.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc7ce0 2024-03-24T12:42:21.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc7e10 2024-03-24T12:42:21.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc7f20 2024-03-24T12:42:21.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc8a40 2024-03-24T12:42:21.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc8ed0 2024-03-24T12:42:21.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc9020 2024-03-24T12:42:21.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc8b50 2024-03-24T12:42:21.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc8ca0 2024-03-24T12:42:21.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc9610 2024-03-24T12:42:21.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc9170 2024-03-24T12:42:21.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc92a0 2024-03-24T12:42:21.840 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc93f0 2024-03-24T12:42:21.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc9740 2024-03-24T12:42:21.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc9bd0 2024-03-24T12:42:21.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc9d00 2024-03-24T12:42:21.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc9850 2024-03-24T12:42:21.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc99a0 2024-03-24T12:42:21.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adca2d0 2024-03-24T12:42:21.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc9e30 2024-03-24T12:42:21.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc9f80 2024-03-24T12:42:21.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adca0b0 2024-03-24T12:42:21.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adca420 2024-03-24T12:42:21.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adca8b0 2024-03-24T12:42:21.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adcad40 2024-03-24T12:42:21.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adcae70 2024-03-24T12:42:21.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adca9c0 2024-03-24T12:42:21.841 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adcab10 2024-03-24T12:42:21.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc8440 2024-03-24T12:42:21.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc8570 2024-03-24T12:42:21.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adcac20 2024-03-24T12:42:21.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adc86a0 2024-03-24T12:42:21.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adcb440 2024-03-24T12:42:21.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adcafa0 2024-03-24T12:42:21.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adcb0f0 2024-03-24T12:42:21.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adcb200 2024-03-24T12:42:21.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adcb330 2024-03-24T12:42:21.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adcba30 2024-03-24T12:42:21.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adcb590 2024-03-24T12:42:21.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adcb6c0 2024-03-24T12:42:21.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: waiting 0x55ac4adcb7d0 2024-03-24T12:42:21.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: done. 2024-03-24T12:42:21.842 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (1811240 ms) 2024-03-24T12:42:21.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (1811240 ms total) 2024-03-24T12:42:21.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: 2024-03-24T12:42:21.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-03-24T12:42:21.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-03-24T12:42:21.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-03-24T12:42:21.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-03-24T12:42:21.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ OK ] LibRadosMisc.Exec (20 ms) 2024-03-24T12:42:21.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-03-24T12:42:21.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (4 ms) 2024-03-24T12:42:21.843 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-03-24T12:42:21.844 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (6 ms) 2024-03-24T12:42:21.844 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-03-24T12:42:22.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:21 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-24T12:42:22.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:21 smithi090 ceph-mon[33646]: osdmap e763: 8 total, 8 up, 8 in 2024-03-24T12:42:22.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:21 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key1"}]: dispatch 2024-03-24T12:42:22.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:21 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key1"}]: dispatch 2024-03-24T12:42:22.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:21 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-24T12:42:22.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:21 smithi006 ceph-mon[30108]: osdmap e763: 8 total, 8 up, 8 in 2024-03-24T12:42:22.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:21 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key1"}]: dispatch 2024-03-24T12:42:22.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:21 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key1"}]: dispatch 2024-03-24T12:42:22.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:21 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-24T12:42:22.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:21 smithi006 ceph-mon[41734]: osdmap e763: 8 total, 8 up, 8 in 2024-03-24T12:42:22.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:21 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/971364581' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key1"}]: dispatch 2024-03-24T12:42:22.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:21 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key1"}]: dispatch 2024-03-24T12:42:23.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:22 smithi090 ceph-mon[33646]: pgmap v1664: 196 pgs: 196 active+clean; 583 KiB data, 897 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:42:23.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key1"}]': finished 2024-03-24T12:42:23.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:22 smithi090 ceph-mon[33646]: osdmap e764: 8 total, 8 up, 8 in 2024-03-24T12:42:23.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:22 smithi006 ceph-mon[30108]: pgmap v1664: 196 pgs: 196 active+clean; 583 KiB data, 897 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:42:23.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key1"}]': finished 2024-03-24T12:42:23.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:22 smithi006 ceph-mon[30108]: osdmap e764: 8 total, 8 up, 8 in 2024-03-24T12:42:23.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:22 smithi006 ceph-mon[41734]: pgmap v1664: 196 pgs: 196 active+clean; 583 KiB data, 897 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:42:23.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi006-81994-1","app":"app1","key":"key1"}]': finished 2024-03-24T12:42:23.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:22 smithi006 ceph-mon[41734]: osdmap e764: 8 total, 8 up, 8 in 2024-03-24T12:42:24.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:42:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:42:25.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:24 smithi090 ceph-mon[33646]: pgmap v1666: 196 pgs: 196 active+clean; 583 KiB data, 890 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T12:42:25.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:24 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:42:25.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:24 smithi006 ceph-mon[30108]: pgmap v1666: 196 pgs: 196 active+clean; 583 KiB data, 890 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T12:42:25.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:24 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:42:25.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:24 smithi006 ceph-mon[41734]: pgmap v1666: 196 pgs: 196 active+clean; 583 KiB data, 890 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T12:42:25.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:24 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:42:25.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:42:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:42:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:42:26.674 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:26 smithi090 ceph-mon[33646]: pgmap v1667: 196 pgs: 196 active+clean; 583 KiB data, 890 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:42:26.707 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:26 smithi006 ceph-mon[30108]: pgmap v1667: 196 pgs: 196 active+clean; 583 KiB data, 890 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:42:26.710 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:26 smithi006 ceph-mon[41734]: pgmap v1667: 196 pgs: 196 active+clean; 583 KiB data, 890 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:42:29.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:28 smithi090 ceph-mon[33646]: pgmap v1668: 196 pgs: 196 active+clean; 583 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T12:42:29.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:28 smithi006 ceph-mon[30108]: pgmap v1668: 196 pgs: 196 active+clean; 583 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T12:42:29.209 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:28 smithi006 ceph-mon[41734]: pgmap v1668: 196 pgs: 196 active+clean; 583 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T12:42:31.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:30 smithi090 ceph-mon[33646]: pgmap v1669: 196 pgs: 196 active+clean; 583 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T12:42:31.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:30 smithi006 ceph-mon[30108]: pgmap v1669: 196 pgs: 196 active+clean; 583 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T12:42:31.209 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:30 smithi006 ceph-mon[41734]: pgmap v1669: 196 pgs: 196 active+clean; 583 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T12:42:33.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:32 smithi090 ceph-mon[33646]: pgmap v1670: 196 pgs: 196 active+clean; 583 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:42:33.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:32 smithi006 ceph-mon[30108]: pgmap v1670: 196 pgs: 196 active+clean; 583 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:42:33.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:32 smithi006 ceph-mon[41734]: pgmap v1670: 196 pgs: 196 active+clean; 583 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:42:34.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:33 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:42:34.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:42:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:42:34.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:33 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:42:34.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:33 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:42:35.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:34 smithi090 ceph-mon[33646]: pgmap v1671: 196 pgs: 196 active+clean; 583 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T12:42:35.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:34 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:42:35.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:42:35.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:34 smithi006 ceph-mon[30108]: pgmap v1671: 196 pgs: 196 active+clean; 583 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T12:42:35.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:34 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:42:35.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:42:35.209 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:34 smithi006 ceph-mon[41734]: pgmap v1671: 196 pgs: 196 active+clean; 583 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T12:42:35.209 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:34 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:42:35.209 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:42:35.709 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:42:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:42:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:42:36.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:35 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:42:36.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:35 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:42:36.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:35 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:42:36.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:35 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:42:36.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:35 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:42:36.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:35 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:42:36.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:35 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:42:36.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:35 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:42:36.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:35 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:42:37.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:36 smithi090 ceph-mon[33646]: pgmap v1672: 196 pgs: 196 active+clean; 583 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:42:37.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:36 smithi006 ceph-mon[30108]: pgmap v1672: 196 pgs: 196 active+clean; 583 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:42:37.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:36 smithi006 ceph-mon[41734]: pgmap v1672: 196 pgs: 196 active+clean; 583 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:42:39.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:38 smithi090 ceph-mon[33646]: pgmap v1673: 196 pgs: 196 active+clean; 583 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:42:39.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:38 smithi006 ceph-mon[30108]: pgmap v1673: 196 pgs: 196 active+clean; 583 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:42:39.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:38 smithi006 ceph-mon[41734]: pgmap v1673: 196 pgs: 196 active+clean; 583 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:42:40.796 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ OK ] LibRadosMisc.Applications (5010 ms) 2024-03-24T12:42:40.796 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-03-24T12:42:40.796 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-03-24T12:42:40.796 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-03-24T12:42:40.796 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-03-24T12:42:40.796 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-03-24T12:42:40.796 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (18174 ms) 2024-03-24T12:42:40.796 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [----------] 8 tests from LibRadosMisc (23214 ms total) 2024-03-24T12:42:40.797 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: 2024-03-24T12:42:40.797 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [----------] Global test environment tear-down 2024-03-24T12:42:40.797 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (1839341 ms total) 2024-03-24T12:42:40.797 INFO:tasks.workunit.client.0.smithi006.stdout: api_misc: [ PASSED ] 12 tests. 2024-03-24T12:42:40.804 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.805 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82080 2024-03-24T12:42:40.805 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82080 2024-03-24T12:42:40.805 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.805 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82587 2024-03-24T12:42:40.805 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82587 2024-03-24T12:42:40.805 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.805 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82614 2024-03-24T12:42:40.805 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82614 2024-03-24T12:42:40.806 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.806 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81861 2024-03-24T12:42:40.806 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 81861 2024-03-24T12:42:40.806 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.806 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82514 2024-03-24T12:42:40.806 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82514 2024-03-24T12:42:40.806 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.806 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82403 2024-03-24T12:42:40.806 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82403 2024-03-24T12:42:40.806 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.806 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81724 2024-03-24T12:42:40.806 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 81724 2024-03-24T12:42:40.806 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.806 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81789 2024-03-24T12:42:40.806 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 81789 2024-03-24T12:42:40.807 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.807 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82425 2024-03-24T12:42:40.807 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82425 2024-03-24T12:42:40.807 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.807 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81715 2024-03-24T12:42:40.807 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 81715 2024-03-24T12:42:40.807 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.807 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81736 2024-03-24T12:42:40.807 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 81736 2024-03-24T12:42:40.807 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.807 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82781 2024-03-24T12:42:40.807 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82781 2024-03-24T12:42:40.807 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.807 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81937 2024-03-24T12:42:40.808 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 81937 2024-03-24T12:42:40.808 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.808 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82891 2024-03-24T12:42:40.808 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82891 2024-03-24T12:42:40.808 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.808 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82054 2024-03-24T12:42:40.808 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82054 2024-03-24T12:42:40.808 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.808 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82559 2024-03-24T12:42:40.808 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82559 2024-03-24T12:42:40.808 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.808 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82989 2024-03-24T12:42:40.808 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82989 2024-03-24T12:42:40.808 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82009 2024-03-24T12:42:40.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82009 2024-03-24T12:42:40.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82443 2024-03-24T12:42:40.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82443 2024-03-24T12:42:40.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82707 2024-03-24T12:42:40.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82707 2024-03-24T12:42:40.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82535 2024-03-24T12:42:40.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82535 2024-03-24T12:42:40.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82833 2024-03-24T12:42:40.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82833 2024-03-24T12:42:40.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.809 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82379 2024-03-24T12:42:40.810 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82379 2024-03-24T12:42:40.810 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.810 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81884 2024-03-24T12:42:40.810 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 81884 2024-03-24T12:42:40.810 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.810 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82975 2024-03-24T12:42:40.810 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82975 2024-03-24T12:42:40.810 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.810 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82233 2024-03-24T12:42:40.810 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82233 2024-03-24T12:42:40.810 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.810 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81708 2024-03-24T12:42:40.810 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 81708 2024-03-24T12:42:40.810 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.811 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81767 2024-03-24T12:42:40.811 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 81767 2024-03-24T12:42:40.811 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.811 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82120 2024-03-24T12:42:40.811 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82120 2024-03-24T12:42:40.811 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.811 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=81807 2024-03-24T12:42:40.811 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 81807 2024-03-24T12:42:40.811 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.811 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82284 2024-03-24T12:42:40.811 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82284 2024-03-24T12:42:40.811 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.811 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82319 2024-03-24T12:42:40.811 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82319 2024-03-24T12:42:40.811 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.812 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=82685 2024-03-24T12:42:40.812 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 82685 2024-03-24T12:42:40.812 INFO:tasks.workunit.client.0.smithi006.stderr:+ for t in "${!pids[@]}" 2024-03-24T12:42:40.812 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=83007 2024-03-24T12:42:40.812 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 83007 2024-03-24T12:42:40.812 INFO:tasks.workunit.client.0.smithi006.stderr:+ exit 0 2024-03-24T12:42:40.812 INFO:tasks.workunit.client.0.smithi006.stderr:+ cleanup 2024-03-24T12:42:40.812 INFO:tasks.workunit.client.0.smithi006.stderr:+ pkill -P 81702 2024-03-24T12:42:40.816 INFO:tasks.workunit.client.0.smithi006.stderr:+ true 2024-03-24T12:42:40.817 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-24T12:42:40.817 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-24T12:42:40.850 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-03-24T12:42:40.851 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-03-24T12:42:40.851 DEBUG:teuthology.orchestra.run.smithi006:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f73f1dfd96849300c4322bd7259a3ab37f94c200 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-03-24T12:42:40.910 INFO:tasks.workunit.client.0.smithi006.stderr:++ uuidgen 2024-03-24T12:42:40.917 INFO:tasks.workunit.client.0.smithi006.stderr:+ p=54e008b8-0c65-4b2f-9ae3-d0f6346347d3 2024-03-24T12:42:40.917 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph osd pool create 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 12 2024-03-24T12:42:41.005 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.006+0000 7fd510247640 1 -- 172.21.15.6:0/4020350918 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd5080692c0 msgr2=0x7fd508105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:42:41.005 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.006+0000 7fd510247640 1 --2- 172.21.15.6:0/4020350918 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd5080692c0 0x7fd508105dc0 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009900 tx=0x7fd4f802bc30 comp rx=0 tx=0).stop 2024-03-24T12:42:41.005 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.006+0000 7fd510247640 1 -- 172.21.15.6:0/4020350918 shutdown_connections 2024-03-24T12:42:41.005 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.006+0000 7fd510247640 1 --2- 172.21.15.6:0/4020350918 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5081064a0 0x7fd50810e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:41.005 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.006+0000 7fd510247640 1 --2- 172.21.15.6:0/4020350918 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd5080692c0 0x7fd508105dc0 unknown :-1 s=CLOSED pgs=3062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:41.005 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.006+0000 7fd510247640 1 --2- 172.21.15.6:0/4020350918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd508068910 0x7fd508068cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:41.005 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.006+0000 7fd510247640 1 -- 172.21.15.6:0/4020350918 >> 172.21.15.6:0/4020350918 conn(0x7fd508077150 msgr2=0x7fd508077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:42:41.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.006+0000 7fd510247640 1 -- 172.21.15.6:0/4020350918 shutdown_connections 2024-03-24T12:42:41.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.006+0000 7fd510247640 1 -- 172.21.15.6:0/4020350918 wait complete. 2024-03-24T12:42:41.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.006+0000 7fd510247640 1 Processor -- start 2024-03-24T12:42:41.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.007+0000 7fd510247640 1 -- start start 2024-03-24T12:42:41.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.007+0000 7fd510247640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd508068910 0x7fd5081000b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:42:41.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.007+0000 7fd510247640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd5081064a0 0x7fd5081005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:42:41.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.007+0000 7fd510247640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd508100b30 0x7fd5081a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:42:41.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.007+0000 7fd510247640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5081a5540 con 0x7fd508068910 2024-03-24T12:42:41.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.007+0000 7fd510247640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5081a56b0 con 0x7fd508100b30 2024-03-24T12:42:41.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.007+0000 7fd510247640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5081a5870 con 0x7fd5081064a0 2024-03-24T12:42:41.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.007+0000 7fd50dfe3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd508068910 0x7fd5081000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:42:41.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.007+0000 7fd50d7e2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd5081064a0 0x7fd5081005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:42:41.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.007+0000 7fd50dfe3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd508068910 0x7fd5081000b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58942/0 (socket says 172.21.15.6:58942) 2024-03-24T12:42:41.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.007+0000 7fd50dfe3640 1 -- 172.21.15.6:0/1043811652 learned_addr learned my addr 172.21.15.6:0/1043811652 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:42:41.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.007+0000 7fd50e7e4640 1 --2- 172.21.15.6:0/1043811652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd508100b30 0x7fd5081a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:42:41.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.007+0000 7fd50dfe3640 1 -- 172.21.15.6:0/1043811652 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd5081064a0 msgr2=0x7fd5081005f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:42:41.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.007+0000 7fd50dfe3640 1 --2- 172.21.15.6:0/1043811652 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd5081064a0 0x7fd5081005f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:41.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.007+0000 7fd50dfe3640 1 -- 172.21.15.6:0/1043811652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd508100b30 msgr2=0x7fd5081a4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:42:41.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.007+0000 7fd50dfe3640 1 --2- 172.21.15.6:0/1043811652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd508100b30 0x7fd5081a4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:41.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.007+0000 7fd50dfe3640 1 -- 172.21.15.6:0/1043811652 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd504009840 con 0x7fd508068910 2024-03-24T12:42:41.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.007+0000 7fd50d7e2640 1 --2- 172.21.15.6:0/1043811652 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd5081064a0 0x7fd5081005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:42:41.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.007+0000 7fd50e7e4640 1 --2- 172.21.15.6:0/1043811652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd508100b30 0x7fd5081a4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T12:42:41.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.007+0000 7fd50dfe3640 1 --2- 172.21.15.6:0/1043811652 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd508068910 0x7fd5081000b0 secure :-1 s=READY pgs=3076 cs=0 l=1 rev1=1 crypto rx=0x7fd50000e9f0 tx=0x7fd50000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:42:41.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.007+0000 7fd4feffd640 1 -- 172.21.15.6:0/1043811652 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd500009720 con 0x7fd508068910 2024-03-24T12:42:41.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.008+0000 7fd4feffd640 1 -- 172.21.15.6:0/1043811652 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd50000cd80 con 0x7fd508068910 2024-03-24T12:42:41.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.008+0000 7fd510247640 1 -- 172.21.15.6:0/1043811652 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f8009560 con 0x7fd508068910 2024-03-24T12:42:41.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.008+0000 7fd4feffd640 1 -- 172.21.15.6:0/1043811652 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd500004ea0 con 0x7fd508068910 2024-03-24T12:42:41.008 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.008+0000 7fd510247640 1 -- 172.21.15.6:0/1043811652 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5081a5e50 con 0x7fd508068910 2024-03-24T12:42:41.008 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.008+0000 7fd510247640 1 -- 172.21.15.6:0/1043811652 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5080681d0 con 0x7fd508068910 2024-03-24T12:42:41.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.009+0000 7fd4feffd640 1 -- 172.21.15.6:0/1043811652 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7fd500005020 con 0x7fd508068910 2024-03-24T12:42:41.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.009+0000 7fd4feffd640 1 --2- 172.21.15.6:0/1043811652 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fd4f406d5d0 0x7fd4f406fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:42:41.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.010+0000 7fd50d7e2640 1 --2- 172.21.15.6:0/1043811652 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fd4f406d5d0 0x7fd4f406fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:42:41.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.010+0000 7fd4feffd640 1 -- 172.21.15.6:0/1043811652 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(765..765 src has 255..765) v4 ==== 8819+0+0 (secure 0 0 0) 0x7fd500014070 con 0x7fd508068910 2024-03-24T12:42:41.010 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.010+0000 7fd50d7e2640 1 --2- 172.21.15.6:0/1043811652 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fd4f406d5d0 0x7fd4f406fa90 secure :-1 s=READY pgs=4097 cs=0 l=1 rev1=1 crypto rx=0x7fd4f802c680 tx=0x7fd4f8005fd0 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:42:41.011 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.015+0000 7fd4feffd640 1 -- 172.21.15.6:0/1043811652 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd500056e60 con 0x7fd508068910 2024-03-24T12:42:41.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:40 smithi090 ceph-mon[33646]: pgmap v1674: 196 pgs: 196 active+clean; 583 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:42:41.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.195+0000 7fd510247640 1 -- 172.21.15.6:0/1043811652 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pg_num": 12} v 0) v1 -- 0x7fd5081a5a30 con 0x7fd508068910 2024-03-24T12:42:41.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:40 smithi006 ceph-mon[30108]: pgmap v1674: 196 pgs: 196 active+clean; 583 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:42:41.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:40 smithi006 ceph-mon[41734]: pgmap v1674: 196 pgs: 196 active+clean; 583 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-24T12:42:41.787 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.791+0000 7fd4feffd640 1 -- 172.21.15.6:0/1043811652 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pg_num": 12}]=0 pool '54e008b8-0c65-4b2f-9ae3-d0f6346347d3' created v766) v1 ==== 176+0+0 (secure 0 0 0) 0x7fd50005a520 con 0x7fd508068910 2024-03-24T12:42:41.903 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.907+0000 7fd510247640 1 -- 172.21.15.6:0/1043811652 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pg_num": 12} v 0) v1 -- 0x7fd508069420 con 0x7fd508068910 2024-03-24T12:42:41.904 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.907+0000 7fd4feffd640 1 -- 172.21.15.6:0/1043811652 <== mon.0 v2:172.21.15.6:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pg_num": 12}]=0 pool '54e008b8-0c65-4b2f-9ae3-d0f6346347d3' already exists v766) v1 ==== 183+0+0 (secure 0 0 0) 0x7fd50005a700 con 0x7fd508068910 2024-03-24T12:42:41.904 INFO:tasks.workunit.client.0.smithi006.stderr:pool '54e008b8-0c65-4b2f-9ae3-d0f6346347d3' already exists 2024-03-24T12:42:41.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.909+0000 7fd510247640 1 -- 172.21.15.6:0/1043811652 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fd4f406d5d0 msgr2=0x7fd4f406fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:42:41.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.909+0000 7fd510247640 1 --2- 172.21.15.6:0/1043811652 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fd4f406d5d0 0x7fd4f406fa90 secure :-1 s=READY pgs=4097 cs=0 l=1 rev1=1 crypto rx=0x7fd4f802c680 tx=0x7fd4f8005fd0 comp rx=0 tx=0).stop 2024-03-24T12:42:41.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.909+0000 7fd510247640 1 -- 172.21.15.6:0/1043811652 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd508068910 msgr2=0x7fd5081000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:42:41.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.909+0000 7fd510247640 1 --2- 172.21.15.6:0/1043811652 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd508068910 0x7fd5081000b0 secure :-1 s=READY pgs=3076 cs=0 l=1 rev1=1 crypto rx=0x7fd50000e9f0 tx=0x7fd50000edf0 comp rx=0 tx=0).stop 2024-03-24T12:42:41.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.909+0000 7fd510247640 1 -- 172.21.15.6:0/1043811652 shutdown_connections 2024-03-24T12:42:41.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.909+0000 7fd510247640 1 --2- 172.21.15.6:0/1043811652 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fd4f406d5d0 0x7fd4f406fa90 unknown :-1 s=CLOSED pgs=4097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:41.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.909+0000 7fd510247640 1 --2- 172.21.15.6:0/1043811652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd508100b30 0x7fd5081a4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:41.906 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.909+0000 7fd510247640 1 --2- 172.21.15.6:0/1043811652 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd5081064a0 0x7fd5081005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:41.907 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.909+0000 7fd510247640 1 --2- 172.21.15.6:0/1043811652 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd508068910 0x7fd5081000b0 unknown :-1 s=CLOSED pgs=3076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:41.907 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.909+0000 7fd510247640 1 -- 172.21.15.6:0/1043811652 >> 172.21.15.6:0/1043811652 conn(0x7fd508077150 msgr2=0x7fd5080fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:42:41.907 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.909+0000 7fd510247640 1 -- 172.21.15.6:0/1043811652 shutdown_connections 2024-03-24T12:42:41.907 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:41.909+0000 7fd510247640 1 -- 172.21.15.6:0/1043811652 wait complete. 2024-03-24T12:42:41.915 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph osd pool set-quota 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 max_objects 10 2024-03-24T12:42:42.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.002+0000 7f654f26b640 1 -- 172.21.15.6:0/3645747010 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f65481017e0 msgr2=0x7f6548104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:42:42.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.002+0000 7f654f26b640 1 --2- 172.21.15.6:0/3645747010 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f65481017e0 0x7f6548104d10 secure :-1 s=READY pgs=3063 cs=0 l=1 rev1=1 crypto rx=0x7f6538009900 tx=0x7f653802bc30 comp rx=0 tx=0).stop 2024-03-24T12:42:42.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.002+0000 7f654f26b640 1 -- 172.21.15.6:0/3645747010 shutdown_connections 2024-03-24T12:42:42.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.002+0000 7f654f26b640 1 --2- 172.21.15.6:0/3645747010 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6548105440 0x7f654810cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:42.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.002+0000 7f654f26b640 1 --2- 172.21.15.6:0/3645747010 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f65481017e0 0x7f6548104d10 unknown :-1 s=CLOSED pgs=3063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:42.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.002+0000 7f654f26b640 1 --2- 172.21.15.6:0/3645747010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6548100e30 0x7f6548101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:42.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.002+0000 7f654f26b640 1 -- 172.21.15.6:0/3645747010 >> 172.21.15.6:0/3645747010 conn(0x7f6548075400 msgr2=0x7f6548075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:42:42.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.002+0000 7f654f26b640 1 -- 172.21.15.6:0/3645747010 shutdown_connections 2024-03-24T12:42:42.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.002+0000 7f654f26b640 1 -- 172.21.15.6:0/3645747010 wait complete. 2024-03-24T12:42:42.002 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.003+0000 7f654f26b640 1 Processor -- start 2024-03-24T12:42:42.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.003+0000 7f654f26b640 1 -- start start 2024-03-24T12:42:42.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.003+0000 7f654f26b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6548100e30 0x7f6548196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:42:42.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.003+0000 7f654f26b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6548105440 0x7f65481972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:42:42.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.003+0000 7f654f26b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f654819b630 0x7f654819bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:42:42.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.003+0000 7f654f26b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654819c020 con 0x7f6548105440 2024-03-24T12:42:42.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.003+0000 7f654f26b640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654819c190 con 0x7f654819b630 2024-03-24T12:42:42.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.003+0000 7f654f26b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f654819c300 con 0x7f6548100e30 2024-03-24T12:42:42.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.003+0000 7f654d007640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6548100e30 0x7f6548196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:42:42.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.003+0000 7f654c806640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6548105440 0x7f65481972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:42:42.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.003+0000 7f654d007640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6548100e30 0x7f6548196d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:45194/0 (socket says 172.21.15.6:45194) 2024-03-24T12:42:42.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.003+0000 7f654d007640 1 -- 172.21.15.6:0/3348438487 learned_addr learned my addr 172.21.15.6:0/3348438487 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:42:42.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.003+0000 7f654d808640 1 --2- 172.21.15.6:0/3348438487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f654819b630 0x7f654819bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:42:42.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.003+0000 7f654d007640 1 -- 172.21.15.6:0/3348438487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f654819b630 msgr2=0x7f654819bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:42:42.003 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.003+0000 7f654d007640 1 --2- 172.21.15.6:0/3348438487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f654819b630 0x7f654819bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:42.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.003+0000 7f654d007640 1 -- 172.21.15.6:0/3348438487 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6548105440 msgr2=0x7f65481972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:42:42.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.003+0000 7f654d007640 1 --2- 172.21.15.6:0/3348438487 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6548105440 0x7f65481972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:42.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.003+0000 7f654d007640 1 -- 172.21.15.6:0/3348438487 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6538009560 con 0x7f6548100e30 2024-03-24T12:42:42.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.003+0000 7f654d808640 1 --2- 172.21.15.6:0/3348438487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f654819b630 0x7f654819bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T12:42:42.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.003+0000 7f654c806640 1 --2- 172.21.15.6:0/3348438487 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6548105440 0x7f65481972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:42:42.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.004+0000 7f654d007640 1 --2- 172.21.15.6:0/3348438487 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6548100e30 0x7f6548196d70 secure :-1 s=READY pgs=3064 cs=0 l=1 rev1=1 crypto rx=0x7f654400e9f0 tx=0x7f654400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:42:42.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.004+0000 7f653e7fc640 1 -- 172.21.15.6:0/3348438487 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544009700 con 0x7f6548100e30 2024-03-24T12:42:42.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.004+0000 7f653e7fc640 1 -- 172.21.15.6:0/3348438487 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f654400cd80 con 0x7f6548100e30 2024-03-24T12:42:42.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.004+0000 7f654f26b640 1 -- 172.21.15.6:0/3348438487 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65481a38e0 con 0x7f6548100e30 2024-03-24T12:42:42.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.004+0000 7f653e7fc640 1 -- 172.21.15.6:0/3348438487 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544018760 con 0x7f6548100e30 2024-03-24T12:42:42.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.004+0000 7f654f26b640 1 -- 172.21.15.6:0/3348438487 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65481a3d60 con 0x7f6548100e30 2024-03-24T12:42:42.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.005+0000 7f654f26b640 1 -- 172.21.15.6:0/3348438487 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f654804ef50 con 0x7f6548100e30 2024-03-24T12:42:42.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.005+0000 7f653e7fc640 1 -- 172.21.15.6:0/3348438487 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7f6544010430 con 0x7f6548100e30 2024-03-24T12:42:42.005 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.006+0000 7f653e7fc640 1 --2- 172.21.15.6:0/3348438487 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f653406d5c0 0x7f653406fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:42:42.005 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.006+0000 7f653e7fc640 1 -- 172.21.15.6:0/3348438487 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(766..766 src has 255..766) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f6544014070 con 0x7f6548100e30 2024-03-24T12:42:42.005 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.006+0000 7f654c806640 1 --2- 172.21.15.6:0/3348438487 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f653406d5c0 0x7f653406fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:42:42.005 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.006+0000 7f654c806640 1 --2- 172.21.15.6:0/3348438487 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f653406d5c0 0x7f653406fa80 secure :-1 s=READY pgs=4098 cs=0 l=1 rev1=1 crypto rx=0x7f6538000c00 tx=0x7f6538005fd0 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:42:42.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.010+0000 7f653e7fc640 1 -- 172.21.15.6:0/3348438487 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6544052d20 con 0x7f6548100e30 2024-03-24T12:42:42.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:41 smithi090 ceph-mon[33646]: osdmap e765: 8 total, 8 up, 8 in 2024-03-24T12:42:42.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:41 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1043811652' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pg_num": 12}]: dispatch 2024-03-24T12:42:42.193 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.196+0000 7f654f26b640 1 -- 172.21.15.6:0/3348438487 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f65481017e0 con 0x7f6548100e30 2024-03-24T12:42:42.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:41 smithi006 ceph-mon[30108]: osdmap e765: 8 total, 8 up, 8 in 2024-03-24T12:42:42.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:41 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1043811652' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pg_num": 12}]: dispatch 2024-03-24T12:42:42.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:41 smithi006 ceph-mon[41734]: osdmap e765: 8 total, 8 up, 8 in 2024-03-24T12:42:42.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:41 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1043811652' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pg_num": 12}]: dispatch 2024-03-24T12:42:42.794 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.797+0000 7f653e7fc640 1 -- 172.21.15.6:0/3348438487 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 v767) v1 ==== 223+0+0 (secure 0 0 0) 0x7f6544094080 con 0x7f6548100e30 2024-03-24T12:42:42.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:42.918+0000 7f654f26b640 1 -- 172.21.15.6:0/3348438487 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f654810f660 con 0x7f6548100e30 2024-03-24T12:42:43.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:42 smithi090 ceph-mon[33646]: pgmap v1676: 164 pgs: 164 active+clean; 582 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 716 B/s rd, 0 op/s 2024-03-24T12:42:43.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:42 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1043811652' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pg_num": 12}]': finished 2024-03-24T12:42:43.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:42 smithi090 ceph-mon[33646]: osdmap e766: 8 total, 8 up, 8 in 2024-03-24T12:42:43.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:42 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1043811652' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pg_num": 12}]: dispatch 2024-03-24T12:42:43.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:42 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3348438487' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:42:43.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:42 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:42:43.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:42 smithi006 ceph-mon[30108]: pgmap v1676: 164 pgs: 164 active+clean; 582 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 716 B/s rd, 0 op/s 2024-03-24T12:42:43.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:42 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1043811652' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pg_num": 12}]': finished 2024-03-24T12:42:43.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:42 smithi006 ceph-mon[30108]: osdmap e766: 8 total, 8 up, 8 in 2024-03-24T12:42:43.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:42 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1043811652' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pg_num": 12}]: dispatch 2024-03-24T12:42:43.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:42 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3348438487' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:42:43.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:42 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:42:43.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:42 smithi006 ceph-mon[41734]: pgmap v1676: 164 pgs: 164 active+clean; 582 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 716 B/s rd, 0 op/s 2024-03-24T12:42:43.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:42 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1043811652' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pg_num": 12}]': finished 2024-03-24T12:42:43.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:42 smithi006 ceph-mon[41734]: osdmap e766: 8 total, 8 up, 8 in 2024-03-24T12:42:43.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:42 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1043811652' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pg_num": 12}]: dispatch 2024-03-24T12:42:43.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:42 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3348438487' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:42:43.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:42 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:42:43.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.809+0000 7f653e7fc640 1 -- 172.21.15.6:0/3348438487 <== mon.2 v2:172.21.15.6:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 v768) v1 ==== 223+0+0 (secure 0 0 0) 0x7f654407a070 con 0x7f6548100e30 2024-03-24T12:42:43.806 INFO:tasks.workunit.client.0.smithi006.stderr:set-quota max_objects = 10 for pool 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 2024-03-24T12:42:43.810 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.812+0000 7f654f26b640 1 -- 172.21.15.6:0/3348438487 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f653406d5c0 msgr2=0x7f653406fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:42:43.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.812+0000 7f654f26b640 1 --2- 172.21.15.6:0/3348438487 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f653406d5c0 0x7f653406fa80 secure :-1 s=READY pgs=4098 cs=0 l=1 rev1=1 crypto rx=0x7f6538000c00 tx=0x7f6538005fd0 comp rx=0 tx=0).stop 2024-03-24T12:42:43.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.812+0000 7f654f26b640 1 -- 172.21.15.6:0/3348438487 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6548100e30 msgr2=0x7f6548196d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:42:43.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.812+0000 7f654f26b640 1 --2- 172.21.15.6:0/3348438487 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6548100e30 0x7f6548196d70 secure :-1 s=READY pgs=3064 cs=0 l=1 rev1=1 crypto rx=0x7f654400e9f0 tx=0x7f654400eeb0 comp rx=0 tx=0).stop 2024-03-24T12:42:43.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.812+0000 7f654f26b640 1 -- 172.21.15.6:0/3348438487 shutdown_connections 2024-03-24T12:42:43.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.812+0000 7f654f26b640 1 --2- 172.21.15.6:0/3348438487 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f653406d5c0 0x7f653406fa80 unknown :-1 s=CLOSED pgs=4098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:43.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.812+0000 7f654f26b640 1 --2- 172.21.15.6:0/3348438487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f654819b630 0x7f654819bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:43.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.812+0000 7f654f26b640 1 --2- 172.21.15.6:0/3348438487 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6548105440 0x7f65481972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:43.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.812+0000 7f654f26b640 1 --2- 172.21.15.6:0/3348438487 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6548100e30 0x7f6548196d70 unknown :-1 s=CLOSED pgs=3064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:43.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.812+0000 7f654f26b640 1 -- 172.21.15.6:0/3348438487 >> 172.21.15.6:0/3348438487 conn(0x7f6548075400 msgr2=0x7f65480ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:42:43.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.812+0000 7f654f26b640 1 -- 172.21.15.6:0/3348438487 shutdown_connections 2024-03-24T12:42:43.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.812+0000 7f654f26b640 1 -- 172.21.15.6:0/3348438487 wait complete. 2024-03-24T12:42:43.820 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph osd pool application enable 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 rados 2024-03-24T12:42:43.912 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.913+0000 7f1215858640 1 -- 172.21.15.6:0/3925152277 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1210101470 msgr2=0x7f1210101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:42:43.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.913+0000 7f1215858640 1 --2- 172.21.15.6:0/3925152277 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1210101470 0x7f1210101850 secure :-1 s=READY pgs=3065 cs=0 l=1 rev1=1 crypto rx=0x7f1200009be0 tx=0x7f120002bef0 comp rx=0 tx=0).stop 2024-03-24T12:42:43.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.913+0000 7f1215858640 1 -- 172.21.15.6:0/3925152277 shutdown_connections 2024-03-24T12:42:43.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.913+0000 7f1215858640 1 --2- 172.21.15.6:0/3925152277 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12101064c0 0x7f121010e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:43.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.913+0000 7f1215858640 1 --2- 172.21.15.6:0/3925152277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1210101e20 0x7f1210105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:43.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.913+0000 7f1215858640 1 --2- 172.21.15.6:0/3925152277 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1210101470 0x7f1210101850 unknown :-1 s=CLOSED pgs=3065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:43.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.913+0000 7f1215858640 1 -- 172.21.15.6:0/3925152277 >> 172.21.15.6:0/3925152277 conn(0x7f12100782d0 msgr2=0x7f12100786e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:42:43.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.914+0000 7f1215858640 1 -- 172.21.15.6:0/3925152277 shutdown_connections 2024-03-24T12:42:43.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.914+0000 7f1215858640 1 -- 172.21.15.6:0/3925152277 wait complete. 2024-03-24T12:42:43.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.914+0000 7f1215858640 1 Processor -- start 2024-03-24T12:42:43.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.914+0000 7f1215858640 1 -- start start 2024-03-24T12:42:43.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.914+0000 7f1215858640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1210101470 0x7f1210198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:42:43.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.914+0000 7f1215858640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12101064c0 0x7f1210199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:42:43.913 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.914+0000 7f1215858640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f121019da90 0x7f121019df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:42:43.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.914+0000 7f1215858640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121019e480 con 0x7f121019da90 2024-03-24T12:42:43.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.914+0000 7f1215858640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121019e5f0 con 0x7f12101064c0 2024-03-24T12:42:43.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.914+0000 7f1215858640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f121019e760 con 0x7f1210101470 2024-03-24T12:42:43.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.915+0000 7f120effd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1210101470 0x7f1210198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:42:43.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.915+0000 7f120f7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f121019da90 0x7f121019df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:42:43.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.915+0000 7f120effd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1210101470 0x7f1210198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:45216/0 (socket says 172.21.15.6:45216) 2024-03-24T12:42:43.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.915+0000 7f120effd640 1 -- 172.21.15.6:0/1942897227 learned_addr learned my addr 172.21.15.6:0/1942897227 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:42:43.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.915+0000 7f120e7fc640 1 --2- 172.21.15.6:0/1942897227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12101064c0 0x7f1210199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:42:43.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.915+0000 7f120effd640 1 -- 172.21.15.6:0/1942897227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12101064c0 msgr2=0x7f1210199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:42:43.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.915+0000 7f120effd640 1 --2- 172.21.15.6:0/1942897227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12101064c0 0x7f1210199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:43.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.915+0000 7f120effd640 1 -- 172.21.15.6:0/1942897227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f121019da90 msgr2=0x7f121019df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:42:43.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.915+0000 7f120effd640 1 --2- 172.21.15.6:0/1942897227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f121019da90 0x7f121019df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:43.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.915+0000 7f120effd640 1 -- 172.21.15.6:0/1942897227 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1208009560 con 0x7f1210101470 2024-03-24T12:42:43.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.915+0000 7f120f7fe640 1 --2- 172.21.15.6:0/1942897227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f121019da90 0x7f121019df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:42:43.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.915+0000 7f120effd640 1 --2- 172.21.15.6:0/1942897227 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1210101470 0x7f1210198fc0 secure :-1 s=READY pgs=3066 cs=0 l=1 rev1=1 crypto rx=0x7f1200000f80 tx=0x7f1200003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:42:43.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.915+0000 7f1214856640 1 -- 172.21.15.6:0/1942897227 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1200033b10 con 0x7f1210101470 2024-03-24T12:42:43.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.915+0000 7f1214856640 1 -- 172.21.15.6:0/1942897227 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1200033c70 con 0x7f1210101470 2024-03-24T12:42:43.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.915+0000 7f1215858640 1 -- 172.21.15.6:0/1942897227 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1200009840 con 0x7f1210101470 2024-03-24T12:42:43.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.915+0000 7f1214856640 1 -- 172.21.15.6:0/1942897227 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1200034690 con 0x7f1210101470 2024-03-24T12:42:43.915 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.915+0000 7f1215858640 1 -- 172.21.15.6:0/1942897227 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12101a6030 con 0x7f1210101470 2024-03-24T12:42:43.916 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.916+0000 7f1215858640 1 -- 172.21.15.6:0/1942897227 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1210078df0 con 0x7f1210101470 2024-03-24T12:42:43.916 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.917+0000 7f1214856640 1 -- 172.21.15.6:0/1942897227 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7f1200004040 con 0x7f1210101470 2024-03-24T12:42:43.916 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.917+0000 7f1214856640 1 --2- 172.21.15.6:0/1942897227 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f11fc06d5d0 0x7f11fc06fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:42:43.916 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.917+0000 7f1214856640 1 -- 172.21.15.6:0/1942897227 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(768..768 src has 255..768) v4 ==== 9194+0+0 (secure 0 0 0) 0x7f12000b08f0 con 0x7f1210101470 2024-03-24T12:42:43.916 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.917+0000 7f120e7fc640 1 --2- 172.21.15.6:0/1942897227 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f11fc06d5d0 0x7f11fc06fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:42:43.916 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.918+0000 7f120e7fc640 1 --2- 172.21.15.6:0/1942897227 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f11fc06d5d0 0x7f11fc06fa90 secure :-1 s=READY pgs=4099 cs=0 l=1 rev1=1 crypto rx=0x7f11f8005fd0 tx=0x7f11f8005850 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:42:43.918 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:43.922+0000 7f1214856640 1 -- 172.21.15.6:0/1942897227 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1200079860 con 0x7f1210101470 2024-03-24T12:42:44.105 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:44.109+0000 7f1215858640 1 -- 172.21.15.6:0/1942897227 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "app": "rados"} v 0) v1 -- 0x7f121004efc0 con 0x7f1210101470 2024-03-24T12:42:44.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:42:43 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:42:44.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:43 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "10"}]': finished 2024-03-24T12:42:44.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:43 smithi090 ceph-mon[33646]: osdmap e767: 8 total, 8 up, 8 in 2024-03-24T12:42:44.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:43 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3348438487' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:42:44.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:43 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:42:44.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:43 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "10"}]': finished 2024-03-24T12:42:44.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:43 smithi006 ceph-mon[30108]: osdmap e767: 8 total, 8 up, 8 in 2024-03-24T12:42:44.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:43 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3348438487' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:42:44.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:43 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:42:44.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:43 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "10"}]': finished 2024-03-24T12:42:44.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:43 smithi006 ceph-mon[41734]: osdmap e767: 8 total, 8 up, 8 in 2024-03-24T12:42:44.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:43 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3348438487' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:42:44.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:43 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:42:44.808 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:44.811+0000 7f1214856640 1 -- 172.21.15.6:0/1942897227 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "app": "rados"}]=0 enabled application 'rados' on pool '54e008b8-0c65-4b2f-9ae3-d0f6346347d3' v769) v1 ==== 213+0+0 (secure 0 0 0) 0x7f120007cf20 con 0x7f1210101470 2024-03-24T12:42:44.924 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:44.927+0000 7f1215858640 1 -- 172.21.15.6:0/1942897227 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "app": "rados"} v 0) v1 -- 0x7f1210101fc0 con 0x7f1210101470 2024-03-24T12:42:45.110 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:44 smithi006 ceph-mon[30108]: pgmap v1679: 176 pgs: 12 creating+peering, 164 active+clean; 582 KiB data, 884 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-24T12:42:45.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:44 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:42:45.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:44 smithi006 ceph-mon[30108]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:42:45.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:44 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "10"}]': finished 2024-03-24T12:42:45.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:44 smithi006 ceph-mon[30108]: osdmap e768: 8 total, 8 up, 8 in 2024-03-24T12:42:45.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:44 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1942897227' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "app": "rados"}]: dispatch 2024-03-24T12:42:45.111 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:44 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "app": "rados"}]: dispatch 2024-03-24T12:42:45.111 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:44 smithi006 ceph-mon[41734]: pgmap v1679: 176 pgs: 12 creating+peering, 164 active+clean; 582 KiB data, 884 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-24T12:42:45.111 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:44 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:42:45.111 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:44 smithi006 ceph-mon[41734]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:42:45.111 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:44 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "10"}]': finished 2024-03-24T12:42:45.111 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:44 smithi006 ceph-mon[41734]: osdmap e768: 8 total, 8 up, 8 in 2024-03-24T12:42:45.112 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:44 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1942897227' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "app": "rados"}]: dispatch 2024-03-24T12:42:45.112 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:44 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "app": "rados"}]: dispatch 2024-03-24T12:42:45.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:44 smithi090 ceph-mon[33646]: pgmap v1679: 176 pgs: 12 creating+peering, 164 active+clean; 582 KiB data, 884 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-24T12:42:45.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:44 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:42:45.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:44 smithi090 ceph-mon[33646]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:42:45.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:44 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "10"}]': finished 2024-03-24T12:42:45.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:44 smithi090 ceph-mon[33646]: osdmap e768: 8 total, 8 up, 8 in 2024-03-24T12:42:45.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:44 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1942897227' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "app": "rados"}]: dispatch 2024-03-24T12:42:45.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:44 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "app": "rados"}]: dispatch 2024-03-24T12:42:45.457 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:42:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:42:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:42:45.820 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:45.823+0000 7f1214856640 1 -- 172.21.15.6:0/1942897227 <== mon.2 v2:172.21.15.6:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "app": "rados"}]=0 enabled application 'rados' on pool '54e008b8-0c65-4b2f-9ae3-d0f6346347d3' v770) v1 ==== 213+0+0 (secure 0 0 0) 0x7f1200043030 con 0x7f1210101470 2024-03-24T12:42:45.820 INFO:tasks.workunit.client.0.smithi006.stderr:enabled application 'rados' on pool '54e008b8-0c65-4b2f-9ae3-d0f6346347d3' 2024-03-24T12:42:45.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:45.826+0000 7f1215858640 1 -- 172.21.15.6:0/1942897227 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f11fc06d5d0 msgr2=0x7f11fc06fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:42:45.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:45.826+0000 7f1215858640 1 --2- 172.21.15.6:0/1942897227 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f11fc06d5d0 0x7f11fc06fa90 secure :-1 s=READY pgs=4099 cs=0 l=1 rev1=1 crypto rx=0x7f11f8005fd0 tx=0x7f11f8005850 comp rx=0 tx=0).stop 2024-03-24T12:42:45.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:45.826+0000 7f1215858640 1 -- 172.21.15.6:0/1942897227 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1210101470 msgr2=0x7f1210198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:42:45.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:45.826+0000 7f1215858640 1 --2- 172.21.15.6:0/1942897227 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1210101470 0x7f1210198fc0 secure :-1 s=READY pgs=3066 cs=0 l=1 rev1=1 crypto rx=0x7f1200000f80 tx=0x7f1200003f10 comp rx=0 tx=0).stop 2024-03-24T12:42:45.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:45.826+0000 7f1215858640 1 -- 172.21.15.6:0/1942897227 shutdown_connections 2024-03-24T12:42:45.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:45.826+0000 7f1215858640 1 --2- 172.21.15.6:0/1942897227 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f11fc06d5d0 0x7f11fc06fa90 unknown :-1 s=CLOSED pgs=4099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:45.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:45.826+0000 7f1215858640 1 --2- 172.21.15.6:0/1942897227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f121019da90 0x7f121019df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:45.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:45.826+0000 7f1215858640 1 --2- 172.21.15.6:0/1942897227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12101064c0 0x7f1210199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:45.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:45.826+0000 7f1215858640 1 --2- 172.21.15.6:0/1942897227 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1210101470 0x7f1210198fc0 unknown :-1 s=CLOSED pgs=3066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:42:45.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:45.826+0000 7f1215858640 1 -- 172.21.15.6:0/1942897227 >> 172.21.15.6:0/1942897227 conn(0x7f12100782d0 msgr2=0x7f12100ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:42:45.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:45.826+0000 7f1215858640 1 -- 172.21.15.6:0/1942897227 shutdown_connections 2024-03-24T12:42:45.824 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:42:45.826+0000 7f1215858640 1 -- 172.21.15.6:0/1942897227 wait complete. 2024-03-24T12:42:45.835 INFO:tasks.workunit.client.0.smithi006.stderr:++ seq 1 10 2024-03-24T12:42:45.843 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in `seq 1 10` 2024-03-24T12:42:45.843 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 put obj1 /etc/passwd 2024-03-24T12:42:45.881 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in `seq 1 10` 2024-03-24T12:42:45.881 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 put obj2 /etc/passwd 2024-03-24T12:42:45.918 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in `seq 1 10` 2024-03-24T12:42:45.918 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 put obj3 /etc/passwd 2024-03-24T12:42:45.953 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in `seq 1 10` 2024-03-24T12:42:45.954 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 put obj4 /etc/passwd 2024-03-24T12:42:45.989 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in `seq 1 10` 2024-03-24T12:42:45.989 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 put obj5 /etc/passwd 2024-03-24T12:42:46.026 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in `seq 1 10` 2024-03-24T12:42:46.026 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 put obj6 /etc/passwd 2024-03-24T12:42:46.060 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in `seq 1 10` 2024-03-24T12:42:46.061 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 put obj7 /etc/passwd 2024-03-24T12:42:46.095 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in `seq 1 10` 2024-03-24T12:42:46.096 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 put obj8 /etc/passwd 2024-03-24T12:42:46.130 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in `seq 1 10` 2024-03-24T12:42:46.131 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 put obj9 /etc/passwd 2024-03-24T12:42:46.165 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in `seq 1 10` 2024-03-24T12:42:46.165 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 put obj10 /etc/passwd 2024-03-24T12:42:46.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "app": "rados"}]': finished 2024-03-24T12:42:46.176 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:45 smithi090 ceph-mon[33646]: osdmap e769: 8 total, 8 up, 8 in 2024-03-24T12:42:46.176 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:45 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1942897227' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "app": "rados"}]: dispatch 2024-03-24T12:42:46.176 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:45 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "app": "rados"}]: dispatch 2024-03-24T12:42:46.200 INFO:tasks.workunit.client.0.smithi006.stderr:+ sleep 30 2024-03-24T12:42:46.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "app": "rados"}]': finished 2024-03-24T12:42:46.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:45 smithi006 ceph-mon[30108]: osdmap e769: 8 total, 8 up, 8 in 2024-03-24T12:42:46.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:45 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1942897227' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "app": "rados"}]: dispatch 2024-03-24T12:42:46.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:45 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "app": "rados"}]: dispatch 2024-03-24T12:42:46.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "app": "rados"}]': finished 2024-03-24T12:42:46.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:45 smithi006 ceph-mon[41734]: osdmap e769: 8 total, 8 up, 8 in 2024-03-24T12:42:46.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:45 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1942897227' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "app": "rados"}]: dispatch 2024-03-24T12:42:46.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:45 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "app": "rados"}]: dispatch 2024-03-24T12:42:47.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:46 smithi090 ceph-mon[33646]: pgmap v1682: 176 pgs: 12 creating+peering, 164 active+clean; 582 KiB data, 884 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:42:47.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:46 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "app": "rados"}]': finished 2024-03-24T12:42:47.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:46 smithi090 ceph-mon[33646]: osdmap e770: 8 total, 8 up, 8 in 2024-03-24T12:42:47.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:46 smithi006 ceph-mon[30108]: pgmap v1682: 176 pgs: 12 creating+peering, 164 active+clean; 582 KiB data, 884 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:42:47.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:46 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "app": "rados"}]': finished 2024-03-24T12:42:47.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:46 smithi006 ceph-mon[30108]: osdmap e770: 8 total, 8 up, 8 in 2024-03-24T12:42:47.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:46 smithi006 ceph-mon[41734]: pgmap v1682: 176 pgs: 12 creating+peering, 164 active+clean; 582 KiB data, 884 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:42:47.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:46 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "app": "rados"}]': finished 2024-03-24T12:42:47.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:46 smithi006 ceph-mon[41734]: osdmap e770: 8 total, 8 up, 8 in 2024-03-24T12:42:49.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:48 smithi090 ceph-mon[33646]: pgmap v1684: 176 pgs: 176 active+clean; 698 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 25 KiB/s wr, 3 op/s 2024-03-24T12:42:49.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:48 smithi006 ceph-mon[30108]: pgmap v1684: 176 pgs: 176 active+clean; 698 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 25 KiB/s wr, 3 op/s 2024-03-24T12:42:49.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:48 smithi006 ceph-mon[41734]: pgmap v1684: 176 pgs: 176 active+clean; 698 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 25 KiB/s wr, 3 op/s 2024-03-24T12:42:50.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:49 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:42:50.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:49 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:42:50.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:49 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:42:51.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:50 smithi090 ceph-mon[33646]: pgmap v1685: 176 pgs: 176 active+clean; 698 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 20 KiB/s wr, 2 op/s 2024-03-24T12:42:51.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:50 smithi006 ceph-mon[30108]: pgmap v1685: 176 pgs: 176 active+clean; 698 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 20 KiB/s wr, 2 op/s 2024-03-24T12:42:51.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:50 smithi006 ceph-mon[41734]: pgmap v1685: 176 pgs: 176 active+clean; 698 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 20 KiB/s wr, 2 op/s 2024-03-24T12:42:52.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:51 smithi090 ceph-mon[33646]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:42:52.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:51 smithi090 ceph-mon[33646]: pool '54e008b8-0c65-4b2f-9ae3-d0f6346347d3' is full (reached quota's max_objects: 10) 2024-03-24T12:42:52.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:51 smithi090 ceph-mon[33646]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-24T12:42:52.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:51 smithi090 ceph-mon[33646]: osdmap e771: 8 total, 8 up, 8 in 2024-03-24T12:42:52.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:51 smithi006 ceph-mon[30108]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:42:52.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:51 smithi006 ceph-mon[30108]: pool '54e008b8-0c65-4b2f-9ae3-d0f6346347d3' is full (reached quota's max_objects: 10) 2024-03-24T12:42:52.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:51 smithi006 ceph-mon[30108]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-24T12:42:52.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:51 smithi006 ceph-mon[30108]: osdmap e771: 8 total, 8 up, 8 in 2024-03-24T12:42:52.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:51 smithi006 ceph-mon[41734]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:42:52.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:51 smithi006 ceph-mon[41734]: pool '54e008b8-0c65-4b2f-9ae3-d0f6346347d3' is full (reached quota's max_objects: 10) 2024-03-24T12:42:52.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:51 smithi006 ceph-mon[41734]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-24T12:42:52.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:51 smithi006 ceph-mon[41734]: osdmap e771: 8 total, 8 up, 8 in 2024-03-24T12:42:53.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:52 smithi090 ceph-mon[33646]: pgmap v1687: 176 pgs: 176 active+clean; 698 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 744 B/s rd, 17 KiB/s wr, 2 op/s 2024-03-24T12:42:53.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:52 smithi006 ceph-mon[30108]: pgmap v1687: 176 pgs: 176 active+clean; 698 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 744 B/s rd, 17 KiB/s wr, 2 op/s 2024-03-24T12:42:53.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:52 smithi006 ceph-mon[41734]: pgmap v1687: 176 pgs: 176 active+clean; 698 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 744 B/s rd, 17 KiB/s wr, 2 op/s 2024-03-24T12:42:54.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:42:53 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:42:55.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:54 smithi090 ceph-mon[33646]: pgmap v1688: 176 pgs: 176 active+clean; 698 KiB data, 875 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-03-24T12:42:55.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:54 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:42:55.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:54 smithi006 ceph-mon[30108]: pgmap v1688: 176 pgs: 176 active+clean; 698 KiB data, 875 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-03-24T12:42:55.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:54 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:42:55.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:54 smithi006 ceph-mon[41734]: pgmap v1688: 176 pgs: 176 active+clean; 698 KiB data, 875 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-03-24T12:42:55.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:54 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:42:55.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:42:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:42:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:42:57.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:56 smithi090 ceph-mon[33646]: pgmap v1689: 176 pgs: 176 active+clean; 698 KiB data, 875 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-03-24T12:42:57.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:56 smithi006 ceph-mon[30108]: pgmap v1689: 176 pgs: 176 active+clean; 698 KiB data, 875 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-03-24T12:42:57.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:56 smithi006 ceph-mon[41734]: pgmap v1689: 176 pgs: 176 active+clean; 698 KiB data, 875 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-03-24T12:42:59.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:42:58 smithi090 ceph-mon[33646]: pgmap v1690: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:42:59.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:42:58 smithi006 ceph-mon[30108]: pgmap v1690: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:42:59.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:42:58 smithi006 ceph-mon[41734]: pgmap v1690: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:43:01.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:00 smithi090 ceph-mon[33646]: pgmap v1691: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:43:01.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:00 smithi006 ceph-mon[30108]: pgmap v1691: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:43:01.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:00 smithi006 ceph-mon[41734]: pgmap v1691: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:43:03.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:02 smithi090 ceph-mon[33646]: pgmap v1692: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 971 B/s rd, 0 op/s 2024-03-24T12:43:03.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:02 smithi006 ceph-mon[30108]: pgmap v1692: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 971 B/s rd, 0 op/s 2024-03-24T12:43:03.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:02 smithi006 ceph-mon[41734]: pgmap v1692: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 971 B/s rd, 0 op/s 2024-03-24T12:43:04.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:43:03 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:43:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:04 smithi090 ceph-mon[33646]: pgmap v1693: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:43:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:04 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:43:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:04 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:43:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:43:05.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:04 smithi006 ceph-mon[30108]: pgmap v1693: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:43:05.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:04 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:43:05.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:04 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:43:05.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:43:05.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:04 smithi006 ceph-mon[41734]: pgmap v1693: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:43:05.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:04 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:43:05.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:04 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:43:05.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:43:05.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:43:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:43:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:43:07.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:06 smithi090 ceph-mon[33646]: pgmap v1694: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:43:07.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:06 smithi006 ceph-mon[30108]: pgmap v1694: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:43:07.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:06 smithi006 ceph-mon[41734]: pgmap v1694: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:43:09.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:08 smithi090 ceph-mon[33646]: pgmap v1695: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:43:09.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:08 smithi006 ceph-mon[30108]: pgmap v1695: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:43:09.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:08 smithi006 ceph-mon[41734]: pgmap v1695: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:43:11.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:10 smithi090 ceph-mon[33646]: pgmap v1696: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:43:11.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:10 smithi006 ceph-mon[30108]: pgmap v1696: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:43:11.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:10 smithi006 ceph-mon[41734]: pgmap v1696: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:43:13.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:12 smithi090 ceph-mon[33646]: pgmap v1697: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:43:13.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:12 smithi006 ceph-mon[30108]: pgmap v1697: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:43:13.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:12 smithi006 ceph-mon[41734]: pgmap v1697: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:43:14.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:43:13 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:43:15.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:14 smithi090 ceph-mon[33646]: pgmap v1698: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:43:15.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:14 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:43:15.209 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:14 smithi006 ceph-mon[30108]: pgmap v1698: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:43:15.209 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:14 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:43:15.209 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:14 smithi006 ceph-mon[41734]: pgmap v1698: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:43:15.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:14 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:43:15.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:43:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:43:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:43:16.201 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=158047 2024-03-24T12:43:16.201 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph osd pool set-quota 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 max_objects 100 2024-03-24T12:43:16.201 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 put onemore /etc/passwd 2024-03-24T12:43:16.297 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.299+0000 7f5131cc0640 1 -- 172.21.15.6:0/4087929618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f512c1002e0 msgr2=0x7f512c1006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:16.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.299+0000 7f5131cc0640 1 --2- 172.21.15.6:0/4087929618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f512c1002e0 0x7f512c1006c0 secure :-1 s=READY pgs=3096 cs=0 l=1 rev1=1 crypto rx=0x7f5114009930 tx=0x7f511402bc20 comp rx=0 tx=0).stop 2024-03-24T12:43:16.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.299+0000 7f5131cc0640 1 -- 172.21.15.6:0/4087929618 shutdown_connections 2024-03-24T12:43:16.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.299+0000 7f5131cc0640 1 --2- 172.21.15.6:0/4087929618 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f512c1053b0 0x7f512c10cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:16.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.299+0000 7f5131cc0640 1 --2- 172.21.15.6:0/4087929618 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f512c100c90 0x7f512c104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:16.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.299+0000 7f5131cc0640 1 --2- 172.21.15.6:0/4087929618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f512c1002e0 0x7f512c1006c0 unknown :-1 s=CLOSED pgs=3096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:16.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.299+0000 7f5131cc0640 1 -- 172.21.15.6:0/4087929618 >> 172.21.15.6:0/4087929618 conn(0x7f512c077150 msgr2=0x7f512c077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:43:16.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.299+0000 7f5131cc0640 1 -- 172.21.15.6:0/4087929618 shutdown_connections 2024-03-24T12:43:16.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.299+0000 7f5131cc0640 1 -- 172.21.15.6:0/4087929618 wait complete. 2024-03-24T12:43:16.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.299+0000 7f5131cc0640 1 Processor -- start 2024-03-24T12:43:16.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.300+0000 7f5131cc0640 1 -- start start 2024-03-24T12:43:16.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.300+0000 7f5131cc0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f512c1002e0 0x7f512c19b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:16.298 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.300+0000 7f5131cc0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f512c1053b0 0x7f512c19be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:16.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.300+0000 7f5131cc0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f512c195ae0 0x7f512c195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:16.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.300+0000 7f5131cc0640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c1964d0 con 0x7f512c195ae0 2024-03-24T12:43:16.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.300+0000 7f5131cc0640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c196640 con 0x7f512c1002e0 2024-03-24T12:43:16.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.300+0000 7f5131cc0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512c196800 con 0x7f512c1053b0 2024-03-24T12:43:16.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.300+0000 7f512affd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f512c1053b0 0x7f512c19be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:16.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.300+0000 7f512bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f512c195ae0 0x7f512c195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:16.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.300+0000 7f512affd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f512c1053b0 0x7f512c19be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:45076/0 (socket says 172.21.15.6:45076) 2024-03-24T12:43:16.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.300+0000 7f512affd640 1 -- 172.21.15.6:0/1319164737 learned_addr learned my addr 172.21.15.6:0/1319164737 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:43:16.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.300+0000 7f512b7fe640 1 --2- 172.21.15.6:0/1319164737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f512c1002e0 0x7f512c19b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:16.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.300+0000 7f512affd640 1 -- 172.21.15.6:0/1319164737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f512c1002e0 msgr2=0x7f512c19b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:16.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.300+0000 7f512affd640 1 --2- 172.21.15.6:0/1319164737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f512c1002e0 0x7f512c19b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:16.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.300+0000 7f512affd640 1 -- 172.21.15.6:0/1319164737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f512c195ae0 msgr2=0x7f512c195f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:16.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.300+0000 7f512affd640 1 --2- 172.21.15.6:0/1319164737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f512c195ae0 0x7f512c195f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:16.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.300+0000 7f512affd640 1 -- 172.21.15.6:0/1319164737 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5124009840 con 0x7f512c1053b0 2024-03-24T12:43:16.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.300+0000 7f512bfff640 1 --2- 172.21.15.6:0/1319164737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f512c195ae0 0x7f512c195f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:43:16.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.300+0000 7f512b7fe640 1 --2- 172.21.15.6:0/1319164737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f512c1002e0 0x7f512c19b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-24T12:43:16.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.300+0000 7f512affd640 1 --2- 172.21.15.6:0/1319164737 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f512c1053b0 0x7f512c19be70 secure :-1 s=READY pgs=3081 cs=0 l=1 rev1=1 crypto rx=0x7f511c00c970 tx=0x7f511c00cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:43:16.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.301+0000 7f5128ff9640 1 -- 172.21.15.6:0/1319164737 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c007bb0 con 0x7f512c1053b0 2024-03-24T12:43:16.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.301+0000 7f5128ff9640 1 -- 172.21.15.6:0/1319164737 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f511c007d10 con 0x7f512c1053b0 2024-03-24T12:43:16.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.301+0000 7f5131cc0640 1 -- 172.21.15.6:0/1319164737 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5114009560 con 0x7f512c1053b0 2024-03-24T12:43:16.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.301+0000 7f5128ff9640 1 -- 172.21.15.6:0/1319164737 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c004bc0 con 0x7f512c1053b0 2024-03-24T12:43:16.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.301+0000 7f5131cc0640 1 -- 172.21.15.6:0/1319164737 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512c1a2b70 con 0x7f512c1053b0 2024-03-24T12:43:16.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.302+0000 7f5131cc0640 1 -- 172.21.15.6:0/1319164737 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512c1017b0 con 0x7f512c1053b0 2024-03-24T12:43:16.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.302+0000 7f5128ff9640 1 -- 172.21.15.6:0/1319164737 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7f511c005020 con 0x7f512c1053b0 2024-03-24T12:43:16.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.303+0000 7f5128ff9640 1 --2- 172.21.15.6:0/1319164737 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f511806d5c0 0x7f511806fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:16.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.303+0000 7f512b7fe640 1 --2- 172.21.15.6:0/1319164737 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f511806d5c0 0x7f511806fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:16.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.303+0000 7f5128ff9640 1 -- 172.21.15.6:0/1319164737 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(771..771 src has 255..771) v4 ==== 9207+0+0 (secure 0 0 0) 0x7f511c08d730 con 0x7f512c1053b0 2024-03-24T12:43:16.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.303+0000 7f5128ff9640 1 -- 172.21.15.6:0/1319164737 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=772}) v3 -- 0x7f5118079260 con 0x7f512c1053b0 2024-03-24T12:43:16.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.303+0000 7f512b7fe640 1 --2- 172.21.15.6:0/1319164737 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f511806d5c0 0x7f511806fa80 secure :-1 s=READY pgs=4111 cs=0 l=1 rev1=1 crypto rx=0x7f5114036040 tx=0x7f5114000bc0 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:43:16.303 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.308+0000 7f5128ff9640 1 -- 172.21.15.6:0/1319164737 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f511c056630 con 0x7f512c1053b0 2024-03-24T12:43:16.484 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.489+0000 7f5131cc0640 1 -- 172.21.15.6:0/1319164737 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f512c196fd0 con 0x7f512c1053b0 2024-03-24T12:43:16.887 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.891+0000 7f5128ff9640 1 -- 172.21.15.6:0/1319164737 <== mon.2 v2:172.21.15.6:3301/0 7 ==== osd_map(772..772 src has 255..772) v4 ==== 628+0+0 (secure 0 0 0) 0x7f511c05aa90 con 0x7f512c1053b0 2024-03-24T12:43:16.887 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.891+0000 7f5128ff9640 1 -- 172.21.15.6:0/1319164737 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=773}) v3 -- 0x7f5118079d90 con 0x7f512c1053b0 2024-03-24T12:43:16.890 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:16.894+0000 7f5128ff9640 1 -- 172.21.15.6:0/1319164737 <== mon.2 v2:172.21.15.6:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 v772) v1 ==== 225+0+0 (secure 0 0 0) 0x7f511c01d070 con 0x7f512c1053b0 2024-03-24T12:43:17.015 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:17.020+0000 7f5131cc0640 1 -- 172.21.15.6:0/1319164737 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f512c100c90 con 0x7f512c1053b0 2024-03-24T12:43:17.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:16 smithi090 ceph-mon[33646]: pgmap v1699: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:43:17.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:16 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1319164737' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "100"}]: dispatch 2024-03-24T12:43:17.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:16 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "100"}]: dispatch 2024-03-24T12:43:17.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:16 smithi006 ceph-mon[30108]: pgmap v1699: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:43:17.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:16 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1319164737' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "100"}]: dispatch 2024-03-24T12:43:17.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:16 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "100"}]: dispatch 2024-03-24T12:43:17.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:16 smithi006 ceph-mon[41734]: pgmap v1699: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:43:17.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:16 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1319164737' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "100"}]: dispatch 2024-03-24T12:43:17.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:16 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "100"}]: dispatch 2024-03-24T12:43:17.892 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:17.895+0000 7f5128ff9640 1 -- 172.21.15.6:0/1319164737 <== mon.2 v2:172.21.15.6:3301/0 9 ==== osd_map(773..773 src has 255..773) v4 ==== 628+0+0 (secure 0 0 0) 0x7f511c05e4e0 con 0x7f512c1053b0 2024-03-24T12:43:17.892 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:17.895+0000 7f5128ff9640 1 -- 172.21.15.6:0/1319164737 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=774}) v3 -- 0x7f511807a310 con 0x7f512c1053b0 2024-03-24T12:43:17.896 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:17.900+0000 7f5128ff9640 1 -- 172.21.15.6:0/1319164737 <== mon.2 v2:172.21.15.6:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 v773) v1 ==== 225+0+0 (secure 0 0 0) 0x7f511c059cf0 con 0x7f512c1053b0 2024-03-24T12:43:17.896 INFO:tasks.workunit.client.0.smithi006.stderr:set-quota max_objects = 100 for pool 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 2024-03-24T12:43:17.900 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:17.904+0000 7f5131cc0640 1 -- 172.21.15.6:0/1319164737 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f511806d5c0 msgr2=0x7f511806fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:17.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:17.904+0000 7f5131cc0640 1 --2- 172.21.15.6:0/1319164737 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f511806d5c0 0x7f511806fa80 secure :-1 s=READY pgs=4111 cs=0 l=1 rev1=1 crypto rx=0x7f5114036040 tx=0x7f5114000bc0 comp rx=0 tx=0).stop 2024-03-24T12:43:17.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:17.904+0000 7f5131cc0640 1 -- 172.21.15.6:0/1319164737 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f512c1053b0 msgr2=0x7f512c19be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:17.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:17.904+0000 7f5131cc0640 1 --2- 172.21.15.6:0/1319164737 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f512c1053b0 0x7f512c19be70 secure :-1 s=READY pgs=3081 cs=0 l=1 rev1=1 crypto rx=0x7f511c00c970 tx=0x7f511c00cd70 comp rx=0 tx=0).stop 2024-03-24T12:43:17.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:17.904+0000 7f5131cc0640 1 -- 172.21.15.6:0/1319164737 shutdown_connections 2024-03-24T12:43:17.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:17.904+0000 7f5131cc0640 1 --2- 172.21.15.6:0/1319164737 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f511806d5c0 0x7f511806fa80 unknown :-1 s=CLOSED pgs=4111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:17.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:17.904+0000 7f5131cc0640 1 --2- 172.21.15.6:0/1319164737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f512c195ae0 0x7f512c195f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:17.901 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:17.904+0000 7f5131cc0640 1 --2- 172.21.15.6:0/1319164737 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f512c1053b0 0x7f512c19be70 unknown :-1 s=CLOSED pgs=3081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:17.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:17.904+0000 7f5131cc0640 1 --2- 172.21.15.6:0/1319164737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f512c1002e0 0x7f512c19b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:17.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:17.904+0000 7f5131cc0640 1 -- 172.21.15.6:0/1319164737 >> 172.21.15.6:0/1319164737 conn(0x7f512c077150 msgr2=0x7f512c0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:43:17.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:17.904+0000 7f5131cc0640 1 -- 172.21.15.6:0/1319164737 shutdown_connections 2024-03-24T12:43:17.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:17.904+0000 7f5131cc0640 1 -- 172.21.15.6:0/1319164737 wait complete. 2024-03-24T12:43:17.912 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 158047 2024-03-24T12:43:18.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "100"}]': finished 2024-03-24T12:43:18.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:17 smithi090 ceph-mon[33646]: osdmap e772: 8 total, 8 up, 8 in 2024-03-24T12:43:18.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:17 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/1319164737' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "100"}]: dispatch 2024-03-24T12:43:18.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:17 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "100"}]: dispatch 2024-03-24T12:43:18.209 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "100"}]': finished 2024-03-24T12:43:18.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:17 smithi006 ceph-mon[30108]: osdmap e772: 8 total, 8 up, 8 in 2024-03-24T12:43:18.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:17 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/1319164737' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "100"}]: dispatch 2024-03-24T12:43:18.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:17 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "100"}]: dispatch 2024-03-24T12:43:18.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "100"}]': finished 2024-03-24T12:43:18.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:17 smithi006 ceph-mon[41734]: osdmap e772: 8 total, 8 up, 8 in 2024-03-24T12:43:18.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:17 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/1319164737' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "100"}]: dispatch 2024-03-24T12:43:18.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:17 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "100"}]: dispatch 2024-03-24T12:43:19.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:18 smithi090 ceph-mon[33646]: pgmap v1701: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:43:19.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:18 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "100"}]': finished 2024-03-24T12:43:19.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:18 smithi090 ceph-mon[33646]: osdmap e773: 8 total, 8 up, 8 in 2024-03-24T12:43:19.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:18 smithi006 ceph-mon[30108]: pgmap v1701: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:43:19.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:18 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "100"}]': finished 2024-03-24T12:43:19.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:18 smithi006 ceph-mon[30108]: osdmap e773: 8 total, 8 up, 8 in 2024-03-24T12:43:19.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:18 smithi006 ceph-mon[41734]: pgmap v1701: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:43:19.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:18 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "100"}]': finished 2024-03-24T12:43:19.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:18 smithi006 ceph-mon[41734]: osdmap e773: 8 total, 8 up, 8 in 2024-03-24T12:43:20.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:43:20.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:19 smithi090 ceph-mon[33646]: pgmap v1703: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:43:20.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:43:20.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:19 smithi006 ceph-mon[30108]: pgmap v1703: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:43:20.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:43:20.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:19 smithi006 ceph-mon[41734]: pgmap v1703: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:43:21.167 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 0 -ne 0 ']' 2024-03-24T12:43:21.168 INFO:tasks.workunit.client.0.smithi006.stderr:+ true 2024-03-24T12:43:21.168 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 put twomore /etc/passwd 2024-03-24T12:43:21.208 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph osd pool set-quota 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 max_bytes 100 2024-03-24T12:43:21.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.301+0000 7f9a1984e640 1 -- 172.21.15.6:0/422016933 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a141053f0 msgr2=0x7f9a1410cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:21.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.301+0000 7f9a1984e640 1 --2- 172.21.15.6:0/422016933 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a141053f0 0x7f9a1410cfc0 secure :-1 s=READY pgs=3099 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c009cd0 tx=0x7f9a0c02c0f0 comp rx=0 tx=0).stop 2024-03-24T12:43:21.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.301+0000 7f9a1984e640 1 -- 172.21.15.6:0/422016933 shutdown_connections 2024-03-24T12:43:21.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.301+0000 7f9a1984e640 1 --2- 172.21.15.6:0/422016933 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a141053f0 0x7f9a1410cfc0 unknown :-1 s=CLOSED pgs=3099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:21.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.301+0000 7f9a1984e640 1 --2- 172.21.15.6:0/422016933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a14100cd0 0x7f9a14104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:21.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.301+0000 7f9a1984e640 1 --2- 172.21.15.6:0/422016933 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a14100320 0x7f9a14100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:21.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.301+0000 7f9a1984e640 1 -- 172.21.15.6:0/422016933 >> 172.21.15.6:0/422016933 conn(0x7f9a14077180 msgr2=0x7f9a14077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:43:21.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.301+0000 7f9a1984e640 1 -- 172.21.15.6:0/422016933 shutdown_connections 2024-03-24T12:43:21.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.301+0000 7f9a1984e640 1 -- 172.21.15.6:0/422016933 wait complete. 2024-03-24T12:43:21.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.302+0000 7f9a1984e640 1 Processor -- start 2024-03-24T12:43:21.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.302+0000 7f9a1984e640 1 -- start start 2024-03-24T12:43:21.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.302+0000 7f9a1984e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a14100320 0x7f9a1410d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:21.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.302+0000 7f9a1984e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a141053f0 0x7f9a1410dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:21.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.302+0000 7f9a1984e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a1410e430 0x7f9a141a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:21.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.302+0000 7f9a1984e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a141a56c0 con 0x7f9a1410e430 2024-03-24T12:43:21.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.302+0000 7f9a1984e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a141a5830 con 0x7f9a14100320 2024-03-24T12:43:21.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.302+0000 7f9a1984e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a141a59f0 con 0x7f9a141053f0 2024-03-24T12:43:21.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.302+0000 7f9a137fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a1410e430 0x7f9a141a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:21.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.302+0000 7f9a0a7fc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a141053f0 0x7f9a1410dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:21.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.302+0000 7f9a0a7fc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a141053f0 0x7f9a1410dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:45114/0 (socket says 172.21.15.6:45114) 2024-03-24T12:43:21.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.302+0000 7f9a0a7fc640 1 -- 172.21.15.6:0/2308913278 learned_addr learned my addr 172.21.15.6:0/2308913278 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:43:21.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.302+0000 7f9a137fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a1410e430 0x7f9a141a4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56578/0 (socket says 172.21.15.6:56578) 2024-03-24T12:43:21.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.302+0000 7f9a12ffd640 1 --2- 172.21.15.6:0/2308913278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a14100320 0x7f9a1410d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:21.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.303+0000 7f9a0a7fc640 1 -- 172.21.15.6:0/2308913278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a14100320 msgr2=0x7f9a1410d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:21.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.303+0000 7f9a0a7fc640 1 --2- 172.21.15.6:0/2308913278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a14100320 0x7f9a1410d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:21.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.303+0000 7f9a0a7fc640 1 -- 172.21.15.6:0/2308913278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a1410e430 msgr2=0x7f9a141a4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:21.301 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.303+0000 7f9a0a7fc640 1 --2- 172.21.15.6:0/2308913278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a1410e430 0x7f9a141a4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:21.304 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.303+0000 7f9a0a7fc640 1 -- 172.21.15.6:0/2308913278 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a04009490 con 0x7f9a141053f0 2024-03-24T12:43:21.304 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.303+0000 7f9a137fe640 1 --2- 172.21.15.6:0/2308913278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a1410e430 0x7f9a141a4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:43:21.304 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.303+0000 7f9a0a7fc640 1 --2- 172.21.15.6:0/2308913278 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a141053f0 0x7f9a1410dde0 secure :-1 s=READY pgs=3083 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00c970 tx=0x7f99fc00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:43:21.304 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.303+0000 7f9a1884c640 1 -- 172.21.15.6:0/2308913278 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc007bb0 con 0x7f9a141053f0 2024-03-24T12:43:21.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.303+0000 7f9a1884c640 1 -- 172.21.15.6:0/2308913278 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f99fc007d10 con 0x7f9a141053f0 2024-03-24T12:43:21.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.303+0000 7f9a1984e640 1 -- 172.21.15.6:0/2308913278 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0c009900 con 0x7f9a141053f0 2024-03-24T12:43:21.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.303+0000 7f9a1884c640 1 -- 172.21.15.6:0/2308913278 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc004cd0 con 0x7f9a141053f0 2024-03-24T12:43:21.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.303+0000 7f9a1984e640 1 -- 172.21.15.6:0/2308913278 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a141a6020 con 0x7f9a141053f0 2024-03-24T12:43:21.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.304+0000 7f9a1984e640 1 -- 172.21.15.6:0/2308913278 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a14077ca0 con 0x7f9a141053f0 2024-03-24T12:43:21.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.304+0000 7f9a1884c640 1 -- 172.21.15.6:0/2308913278 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7f99fc004e70 con 0x7f9a141053f0 2024-03-24T12:43:21.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.305+0000 7f9a1884c640 1 --2- 172.21.15.6:0/2308913278 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f9a0006d5d0 0x7f9a0006fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:21.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.305+0000 7f9a12ffd640 1 --2- 172.21.15.6:0/2308913278 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f9a0006d5d0 0x7f9a0006fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:21.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.305+0000 7f9a1884c640 1 -- 172.21.15.6:0/2308913278 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(774..774 src has 255..774) v4 ==== 9207+0+0 (secure 0 0 0) 0x7f99fc08d750 con 0x7f9a141053f0 2024-03-24T12:43:21.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.305+0000 7f9a12ffd640 1 --2- 172.21.15.6:0/2308913278 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f9a0006d5d0 0x7f9a0006fa90 secure :-1 s=READY pgs=4113 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009d60 tx=0x7f9a04005f90 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:43:21.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.310+0000 7f9a1884c640 1 -- 172.21.15.6:0/2308913278 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f99fc056760 con 0x7f9a141053f0 2024-03-24T12:43:21.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:21 smithi006 ceph-mon[30108]: pool '54e008b8-0c65-4b2f-9ae3-d0f6346347d3' no longer out of quota; removing NO_QUOTA flag 2024-03-24T12:43:21.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:21 smithi006 ceph-mon[30108]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-24T12:43:21.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:21 smithi006 ceph-mon[30108]: osdmap e774: 8 total, 8 up, 8 in 2024-03-24T12:43:21.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:21 smithi006 ceph-mon[41734]: pool '54e008b8-0c65-4b2f-9ae3-d0f6346347d3' no longer out of quota; removing NO_QUOTA flag 2024-03-24T12:43:21.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:21 smithi006 ceph-mon[41734]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-24T12:43:21.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:21 smithi006 ceph-mon[41734]: osdmap e774: 8 total, 8 up, 8 in 2024-03-24T12:43:21.487 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:21.491+0000 7f9a1984e640 1 -- 172.21.15.6:0/2308913278 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f9a1404efc0 con 0x7f9a141053f0 2024-03-24T12:43:21.674 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:21 smithi090 ceph-mon[33646]: pool '54e008b8-0c65-4b2f-9ae3-d0f6346347d3' no longer out of quota; removing NO_QUOTA flag 2024-03-24T12:43:21.675 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:21 smithi090 ceph-mon[33646]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-24T12:43:21.675 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:21 smithi090 ceph-mon[33646]: osdmap e774: 8 total, 8 up, 8 in 2024-03-24T12:43:22.226 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:22.228+0000 7f9a1884c640 1 -- 172.21.15.6:0/2308913278 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 v775) v1 ==== 221+0+0 (secure 0 0 0) 0x7f99fc059e20 con 0x7f9a141053f0 2024-03-24T12:43:22.334 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:22.339+0000 7f9a1984e640 1 -- 172.21.15.6:0/2308913278 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f9a14108680 con 0x7f9a141053f0 2024-03-24T12:43:22.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:22 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2308913278' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-24T12:43:22.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:22 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-24T12:43:22.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:22 smithi006 ceph-mon[30108]: pgmap v1705: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:43:22.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:22 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2308913278' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-24T12:43:22.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:22 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-24T12:43:22.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:22 smithi006 ceph-mon[41734]: pgmap v1705: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:43:22.674 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:22 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2308913278' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-24T12:43:22.675 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:22 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-24T12:43:22.675 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:22 smithi090 ceph-mon[33646]: pgmap v1705: 176 pgs: 176 active+clean; 698 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:43:23.229 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:23.233+0000 7f9a1884c640 1 -- 172.21.15.6:0/2308913278 <== mon.2 v2:172.21.15.6:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 v776) v1 ==== 221+0+0 (secure 0 0 0) 0x7f99fc01d020 con 0x7f9a141053f0 2024-03-24T12:43:23.229 INFO:tasks.workunit.client.0.smithi006.stderr:set-quota max_bytes = 100 for pool 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 2024-03-24T12:43:23.232 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:23.236+0000 7f9a1984e640 1 -- 172.21.15.6:0/2308913278 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f9a0006d5d0 msgr2=0x7f9a0006fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:23.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:23.236+0000 7f9a1984e640 1 --2- 172.21.15.6:0/2308913278 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f9a0006d5d0 0x7f9a0006fa90 secure :-1 s=READY pgs=4113 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009d60 tx=0x7f9a04005f90 comp rx=0 tx=0).stop 2024-03-24T12:43:23.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:23.236+0000 7f9a1984e640 1 -- 172.21.15.6:0/2308913278 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a141053f0 msgr2=0x7f9a1410dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:23.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:23.236+0000 7f9a1984e640 1 --2- 172.21.15.6:0/2308913278 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a141053f0 0x7f9a1410dde0 secure :-1 s=READY pgs=3083 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00c970 tx=0x7f99fc00ce30 comp rx=0 tx=0).stop 2024-03-24T12:43:23.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:23.237+0000 7f9a1984e640 1 -- 172.21.15.6:0/2308913278 shutdown_connections 2024-03-24T12:43:23.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:23.237+0000 7f9a1984e640 1 --2- 172.21.15.6:0/2308913278 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f9a0006d5d0 0x7f9a0006fa90 unknown :-1 s=CLOSED pgs=4113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:23.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:23.237+0000 7f9a1984e640 1 --2- 172.21.15.6:0/2308913278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9a1410e430 0x7f9a141a4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:23.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:23.237+0000 7f9a1984e640 1 --2- 172.21.15.6:0/2308913278 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9a141053f0 0x7f9a1410dde0 unknown :-1 s=CLOSED pgs=3083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:23.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:23.237+0000 7f9a1984e640 1 --2- 172.21.15.6:0/2308913278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a14100320 0x7f9a1410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:23.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:23.237+0000 7f9a1984e640 1 -- 172.21.15.6:0/2308913278 >> 172.21.15.6:0/2308913278 conn(0x7f9a14077180 msgr2=0x7f9a140fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:43:23.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:23.237+0000 7f9a1984e640 1 -- 172.21.15.6:0/2308913278 shutdown_connections 2024-03-24T12:43:23.233 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:23.237+0000 7f9a1984e640 1 -- 172.21.15.6:0/2308913278 wait complete. 2024-03-24T12:43:23.246 INFO:tasks.workunit.client.0.smithi006.stderr:+ sleep 30 2024-03-24T12:43:23.675 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "100"}]': finished 2024-03-24T12:43:23.675 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:23 smithi090 ceph-mon[33646]: osdmap e775: 8 total, 8 up, 8 in 2024-03-24T12:43:23.675 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:23 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2308913278' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-24T12:43:23.675 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:23 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-24T12:43:23.708 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "100"}]': finished 2024-03-24T12:43:23.708 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:23 smithi006 ceph-mon[41734]: osdmap e775: 8 total, 8 up, 8 in 2024-03-24T12:43:23.708 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:23 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2308913278' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-24T12:43:23.708 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:23 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-24T12:43:23.708 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "100"}]': finished 2024-03-24T12:43:23.708 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:23 smithi006 ceph-mon[30108]: osdmap e775: 8 total, 8 up, 8 in 2024-03-24T12:43:23.708 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:23 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2308913278' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-24T12:43:23.708 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:23 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-24T12:43:24.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:43:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:43:24.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:24 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "100"}]': finished 2024-03-24T12:43:24.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:24 smithi090 ceph-mon[33646]: osdmap e776: 8 total, 8 up, 8 in 2024-03-24T12:43:24.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:24 smithi090 ceph-mon[33646]: pgmap v1708: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 882 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-03-24T12:43:24.678 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:24 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:43:24.707 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:24 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "100"}]': finished 2024-03-24T12:43:24.707 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:24 smithi006 ceph-mon[30108]: osdmap e776: 8 total, 8 up, 8 in 2024-03-24T12:43:24.707 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:24 smithi006 ceph-mon[30108]: pgmap v1708: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 882 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-03-24T12:43:24.707 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:24 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:43:24.708 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:24 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "100"}]': finished 2024-03-24T12:43:24.708 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:24 smithi006 ceph-mon[41734]: osdmap e776: 8 total, 8 up, 8 in 2024-03-24T12:43:24.708 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:24 smithi006 ceph-mon[41734]: pgmap v1708: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 882 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-03-24T12:43:24.708 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:24 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:43:25.708 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:43:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:43:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:43:27.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:26 smithi090 ceph-mon[33646]: pgmap v1709: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-03-24T12:43:27.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:26 smithi090 ceph-mon[33646]: pool '54e008b8-0c65-4b2f-9ae3-d0f6346347d3' is full (reached quota's max_bytes: 100 B) 2024-03-24T12:43:27.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:26 smithi090 ceph-mon[33646]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-24T12:43:27.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:26 smithi090 ceph-mon[33646]: osdmap e777: 8 total, 8 up, 8 in 2024-03-24T12:43:27.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:26 smithi006 ceph-mon[30108]: pgmap v1709: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-03-24T12:43:27.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:26 smithi006 ceph-mon[30108]: pool '54e008b8-0c65-4b2f-9ae3-d0f6346347d3' is full (reached quota's max_bytes: 100 B) 2024-03-24T12:43:27.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:26 smithi006 ceph-mon[30108]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-24T12:43:27.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:26 smithi006 ceph-mon[30108]: osdmap e777: 8 total, 8 up, 8 in 2024-03-24T12:43:27.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:26 smithi006 ceph-mon[41734]: pgmap v1709: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-03-24T12:43:27.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:26 smithi006 ceph-mon[41734]: pool '54e008b8-0c65-4b2f-9ae3-d0f6346347d3' is full (reached quota's max_bytes: 100 B) 2024-03-24T12:43:27.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:26 smithi006 ceph-mon[41734]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-24T12:43:27.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:26 smithi006 ceph-mon[41734]: osdmap e777: 8 total, 8 up, 8 in 2024-03-24T12:43:29.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:28 smithi090 ceph-mon[33646]: pgmap v1711: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-03-24T12:43:29.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:28 smithi006 ceph-mon[30108]: pgmap v1711: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-03-24T12:43:29.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:28 smithi006 ceph-mon[41734]: pgmap v1711: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-03-24T12:43:31.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:30 smithi090 ceph-mon[33646]: pgmap v1712: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 684 B/s rd, 0 op/s 2024-03-24T12:43:31.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:30 smithi006 ceph-mon[30108]: pgmap v1712: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 684 B/s rd, 0 op/s 2024-03-24T12:43:31.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:30 smithi006 ceph-mon[41734]: pgmap v1712: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 684 B/s rd, 0 op/s 2024-03-24T12:43:33.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:32 smithi090 ceph-mon[33646]: pgmap v1713: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 604 B/s rd, 0 op/s 2024-03-24T12:43:33.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:32 smithi006 ceph-mon[30108]: pgmap v1713: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 604 B/s rd, 0 op/s 2024-03-24T12:43:33.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:32 smithi006 ceph-mon[41734]: pgmap v1713: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 604 B/s rd, 0 op/s 2024-03-24T12:43:34.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:43:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:43:35.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:34 smithi090 ceph-mon[33646]: pgmap v1714: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:43:35.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:34 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:43:35.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:43:35.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:34 smithi006 ceph-mon[30108]: pgmap v1714: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:43:35.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:34 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:43:35.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:43:35.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:34 smithi006 ceph-mon[41734]: pgmap v1714: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:43:35.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:34 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:43:35.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:43:35.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:43:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:43:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:43:36.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:35 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:43:36.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:35 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:43:36.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:35 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:43:36.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:36 smithi006 ceph-mon[30108]: pgmap v1715: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:43:36.957 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:36 smithi006 ceph-mon[41734]: pgmap v1715: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:43:37.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:36 smithi090 ceph-mon[33646]: pgmap v1715: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:43:39.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:38 smithi090 ceph-mon[33646]: pgmap v1716: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T12:43:39.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:38 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:43:39.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:38 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:43:39.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:38 smithi006 ceph-mon[30108]: pgmap v1716: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T12:43:39.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:38 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:43:39.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:38 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:43:39.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:38 smithi006 ceph-mon[41734]: pgmap v1716: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T12:43:39.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:38 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:43:39.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:38 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:43:41.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:40 smithi090 ceph-mon[33646]: pgmap v1717: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:43:41.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:40 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:43:41.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:40 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:43:41.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:40 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:43:41.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:40 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:43:41.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:40 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:43:41.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:40 smithi006 ceph-mon[30108]: pgmap v1717: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:43:41.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:40 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:43:41.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:40 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:43:41.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:40 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:43:41.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:40 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:43:41.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:40 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:43:41.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:40 smithi006 ceph-mon[41734]: pgmap v1717: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:43:41.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:40 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:43:41.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:40 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:43:41.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:40 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:43:41.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:40 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:43:41.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:40 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:43:43.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:42 smithi090 ceph-mon[33646]: pgmap v1718: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:43:43.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:42 smithi006 ceph-mon[30108]: pgmap v1718: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:43:43.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:42 smithi006 ceph-mon[41734]: pgmap v1718: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:43:44.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:43:43 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:43:45.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:44 smithi090 ceph-mon[33646]: pgmap v1719: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:43:45.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:44 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:43:45.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:44 smithi006 ceph-mon[30108]: pgmap v1719: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:43:45.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:44 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:43:45.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:44 smithi006 ceph-mon[41734]: pgmap v1719: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:43:45.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:44 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:43:45.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:43:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:43:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:43:47.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:46 smithi090 ceph-mon[33646]: pgmap v1720: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:43:47.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:46 smithi006 ceph-mon[30108]: pgmap v1720: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:43:47.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:46 smithi006 ceph-mon[41734]: pgmap v1720: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:43:49.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:48 smithi090 ceph-mon[33646]: pgmap v1721: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:43:49.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:48 smithi006 ceph-mon[30108]: pgmap v1721: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:43:49.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:48 smithi006 ceph-mon[41734]: pgmap v1721: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:43:50.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:49 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:43:50.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:49 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:43:50.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:49 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:43:51.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:50 smithi090 ceph-mon[33646]: pgmap v1722: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:43:51.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:50 smithi006 ceph-mon[30108]: pgmap v1722: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:43:51.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:50 smithi006 ceph-mon[41734]: pgmap v1722: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:43:52.424 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:52 smithi090 ceph-mon[33646]: pgmap v1723: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:43:52.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:52 smithi006 ceph-mon[30108]: pgmap v1723: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:43:52.457 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:52 smithi006 ceph-mon[41734]: pgmap v1723: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:43:53.246 INFO:tasks.workunit.client.0.smithi006.stderr:+ pid=158757 2024-03-24T12:43:53.246 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph osd pool set-quota 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 max_bytes 0 2024-03-24T12:43:53.246 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 put two /etc/passwd 2024-03-24T12:43:53.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.347+0000 7f37227bc640 1 -- 172.21.15.6:0/3417375386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f371c1053c0 msgr2=0x7f371c10cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:53.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.347+0000 7f37227bc640 1 --2- 172.21.15.6:0/3417375386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f371c1053c0 0x7f371c10cf90 secure :-1 s=READY pgs=3101 cs=0 l=1 rev1=1 crypto rx=0x7f3714009c10 tx=0x7f371402bef0 comp rx=0 tx=0).stop 2024-03-24T12:43:53.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.347+0000 7f37227bc640 1 -- 172.21.15.6:0/3417375386 shutdown_connections 2024-03-24T12:43:53.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.347+0000 7f37227bc640 1 --2- 172.21.15.6:0/3417375386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f371c1053c0 0x7f371c10cf90 unknown :-1 s=CLOSED pgs=3101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:53.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.347+0000 7f37227bc640 1 --2- 172.21.15.6:0/3417375386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f371c100ca0 0x7f371c104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:53.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.347+0000 7f37227bc640 1 --2- 172.21.15.6:0/3417375386 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f371c1002f0 0x7f371c1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:53.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.347+0000 7f37227bc640 1 -- 172.21.15.6:0/3417375386 >> 172.21.15.6:0/3417375386 conn(0x7f371c076400 msgr2=0x7f371c076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:43:53.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.347+0000 7f37227bc640 1 -- 172.21.15.6:0/3417375386 shutdown_connections 2024-03-24T12:43:53.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.347+0000 7f37227bc640 1 -- 172.21.15.6:0/3417375386 wait complete. 2024-03-24T12:43:53.349 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.347+0000 7f37227bc640 1 Processor -- start 2024-03-24T12:43:53.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.347+0000 7f37227bc640 1 -- start start 2024-03-24T12:43:53.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.348+0000 7f37227bc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f371c1002f0 0x7f371c10d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:53.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.348+0000 7f37227bc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f371c1053c0 0x7f371c10ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:53.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.348+0000 7f37227bc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f371c10e400 0x7f371c1a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:53.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.348+0000 7f37227bc640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c1a56c0 con 0x7f371c10e400 2024-03-24T12:43:53.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.348+0000 7f37227bc640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c1a5830 con 0x7f371c1053c0 2024-03-24T12:43:53.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.348+0000 7f37227bc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f371c1a59f0 con 0x7f371c1002f0 2024-03-24T12:43:53.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.348+0000 7f371bfff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f371c1002f0 0x7f371c10d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:53.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.348+0000 7f3720d59640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f371c10e400 0x7f371c1a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:53.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.348+0000 7f371bfff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f371c1002f0 0x7f371c10d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:43474/0 (socket says 172.21.15.6:43474) 2024-03-24T12:43:53.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.348+0000 7f3720d59640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f371c10e400 0x7f371c1a4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53240/0 (socket says 172.21.15.6:53240) 2024-03-24T12:43:53.350 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.348+0000 7f371bfff640 1 -- 172.21.15.6:0/2193507516 learned_addr learned my addr 172.21.15.6:0/2193507516 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:43:53.351 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.348+0000 7f371b7fe640 1 --2- 172.21.15.6:0/2193507516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f371c1053c0 0x7f371c10ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:53.351 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.348+0000 7f371bfff640 1 -- 172.21.15.6:0/2193507516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f371c1053c0 msgr2=0x7f371c10ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:53.351 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.348+0000 7f371bfff640 1 --2- 172.21.15.6:0/2193507516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f371c1053c0 0x7f371c10ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:53.351 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.348+0000 7f371bfff640 1 -- 172.21.15.6:0/2193507516 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f371c10e400 msgr2=0x7f371c1a4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:53.351 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.348+0000 7f371bfff640 1 --2- 172.21.15.6:0/2193507516 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f371c10e400 0x7f371c1a4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:53.351 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.348+0000 7f371bfff640 1 -- 172.21.15.6:0/2193507516 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3704009560 con 0x7f371c1002f0 2024-03-24T12:43:53.351 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.348+0000 7f3720d59640 1 --2- 172.21.15.6:0/2193507516 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f371c10e400 0x7f371c1a4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:43:53.351 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.348+0000 7f371bfff640 1 --2- 172.21.15.6:0/2193507516 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f371c1002f0 0x7f371c10d870 secure :-1 s=READY pgs=3086 cs=0 l=1 rev1=1 crypto rx=0x7f370400e9c0 tx=0x7f370400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:43:53.351 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.349+0000 7f37197fa640 1 -- 172.21.15.6:0/2193507516 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370400cde0 con 0x7f371c1002f0 2024-03-24T12:43:53.351 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.349+0000 7f37227bc640 1 -- 172.21.15.6:0/2193507516 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3714009840 con 0x7f371c1002f0 2024-03-24T12:43:53.351 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.349+0000 7f37197fa640 1 -- 172.21.15.6:0/2193507516 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f37040044a0 con 0x7f371c1002f0 2024-03-24T12:43:53.351 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.349+0000 7f37197fa640 1 -- 172.21.15.6:0/2193507516 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3704010430 con 0x7f371c1002f0 2024-03-24T12:43:53.351 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.349+0000 7f37227bc640 1 -- 172.21.15.6:0/2193507516 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f371c1a6020 con 0x7f371c1002f0 2024-03-24T12:43:53.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.350+0000 7f37227bc640 1 -- 172.21.15.6:0/2193507516 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f371c076f20 con 0x7f371c1002f0 2024-03-24T12:43:53.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.350+0000 7f37197fa640 1 -- 172.21.15.6:0/2193507516 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7f3704003ee0 con 0x7f371c1002f0 2024-03-24T12:43:53.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.351+0000 7f37197fa640 1 --2- 172.21.15.6:0/2193507516 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f370806d5d0 0x7f370806fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:53.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.351+0000 7f37197fa640 1 -- 172.21.15.6:0/2193507516 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(777..777 src has 255..777) v4 ==== 9207+0+0 (secure 0 0 0) 0x7f3704014070 con 0x7f371c1002f0 2024-03-24T12:43:53.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.351+0000 7f371b7fe640 1 --2- 172.21.15.6:0/2193507516 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f370806d5d0 0x7f370806fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:53.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.351+0000 7f37197fa640 1 -- 172.21.15.6:0/2193507516 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=778}) v3 -- 0x7f3708079270 con 0x7f371c1002f0 2024-03-24T12:43:53.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.351+0000 7f371b7fe640 1 --2- 172.21.15.6:0/2193507516 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f370806d5d0 0x7f370806fa90 secure :-1 s=READY pgs=4115 cs=0 l=1 rev1=1 crypto rx=0x7f371c10f510 tx=0x7f370c00b410 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:43:53.352 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.356+0000 7f37197fa640 1 -- 172.21.15.6:0/2193507516 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f370407a020 con 0x7f371c1002f0 2024-03-24T12:43:53.536 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.540+0000 7f37227bc640 1 -- 172.21.15.6:0/2193507516 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f371c04efc0 con 0x7f371c1002f0 2024-03-24T12:43:53.590 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.594+0000 7f37197fa640 1 -- 172.21.15.6:0/2193507516 <== mon.2 v2:172.21.15.6:3301/0 7 ==== osd_map(778..778 src has 255..778) v4 ==== 628+0+0 (secure 0 0 0) 0x7f370405ebe0 con 0x7f371c1002f0 2024-03-24T12:43:53.590 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.594+0000 7f37197fa640 1 -- 172.21.15.6:0/2193507516 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=779}) v3 -- 0x7f3708079da0 con 0x7f371c1002f0 2024-03-24T12:43:53.595 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.599+0000 7f37197fa640 1 -- 172.21.15.6:0/2193507516 <== mon.2 v2:172.21.15.6:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 v778) v1 ==== 217+0+0 (secure 0 0 0) 0x7f3704056de0 con 0x7f371c1002f0 2024-03-24T12:43:53.705 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:53.710+0000 7f37227bc640 1 -- 172.21.15.6:0/2193507516 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f371c100ca0 con 0x7f371c1002f0 2024-03-24T12:43:53.852 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:53 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2193507516' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:43:53.853 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:53 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:43:53.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:53 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2193507516' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:43:53.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:53 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:43:53.957 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:53 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2193507516' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:43:53.957 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:53 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:43:54.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:43:53 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:43:54.606 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.610+0000 7f37197fa640 1 -- 172.21.15.6:0/2193507516 <== mon.2 v2:172.21.15.6:3301/0 9 ==== osd_map(779..779 src has 255..779) v4 ==== 628+0+0 (secure 0 0 0) 0x7f37040631a0 con 0x7f371c1002f0 2024-03-24T12:43:54.606 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.610+0000 7f37197fa640 1 -- 172.21.15.6:0/2193507516 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=780}) v3 -- 0x7f370807a190 con 0x7f371c1002f0 2024-03-24T12:43:54.611 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.615+0000 7f37197fa640 1 -- 172.21.15.6:0/2193507516 <== mon.2 v2:172.21.15.6:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 v779) v1 ==== 217+0+0 (secure 0 0 0) 0x7f370405a910 con 0x7f371c1002f0 2024-03-24T12:43:54.611 INFO:tasks.workunit.client.0.smithi006.stderr:set-quota max_bytes = 0 for pool 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 2024-03-24T12:43:54.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.619+0000 7f37227bc640 1 -- 172.21.15.6:0/2193507516 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f370806d5d0 msgr2=0x7f370806fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:54.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.619+0000 7f37227bc640 1 --2- 172.21.15.6:0/2193507516 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f370806d5d0 0x7f370806fa90 secure :-1 s=READY pgs=4115 cs=0 l=1 rev1=1 crypto rx=0x7f371c10f510 tx=0x7f370c00b410 comp rx=0 tx=0).stop 2024-03-24T12:43:54.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.619+0000 7f37227bc640 1 -- 172.21.15.6:0/2193507516 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f371c1002f0 msgr2=0x7f371c10d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:54.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.619+0000 7f37227bc640 1 --2- 172.21.15.6:0/2193507516 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f371c1002f0 0x7f371c10d870 secure :-1 s=READY pgs=3086 cs=0 l=1 rev1=1 crypto rx=0x7f370400e9c0 tx=0x7f370400ee80 comp rx=0 tx=0).stop 2024-03-24T12:43:54.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.619+0000 7f37227bc640 1 -- 172.21.15.6:0/2193507516 shutdown_connections 2024-03-24T12:43:54.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.619+0000 7f37227bc640 1 --2- 172.21.15.6:0/2193507516 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f370806d5d0 0x7f370806fa90 unknown :-1 s=CLOSED pgs=4115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:54.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.619+0000 7f37227bc640 1 --2- 172.21.15.6:0/2193507516 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f371c10e400 0x7f371c1a4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:54.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.619+0000 7f37227bc640 1 --2- 172.21.15.6:0/2193507516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f371c1053c0 0x7f371c10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:54.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.619+0000 7f37227bc640 1 --2- 172.21.15.6:0/2193507516 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f371c1002f0 0x7f371c10d870 unknown :-1 s=CLOSED pgs=3086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:54.615 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.619+0000 7f37227bc640 1 -- 172.21.15.6:0/2193507516 >> 172.21.15.6:0/2193507516 conn(0x7f371c076400 msgr2=0x7f371c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:43:54.616 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.619+0000 7f37227bc640 1 -- 172.21.15.6:0/2193507516 shutdown_connections 2024-03-24T12:43:54.616 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.619+0000 7f37227bc640 1 -- 172.21.15.6:0/2193507516 wait complete. 2024-03-24T12:43:54.626 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph osd pool set-quota 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 max_objects 0 2024-03-24T12:43:54.716 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.718+0000 7fbfea822640 1 -- 172.21.15.6:0/3480114209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfe4100380 msgr2=0x7fbfe4100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:54.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.718+0000 7fbfea822640 1 --2- 172.21.15.6:0/3480114209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfe4100380 0x7fbfe4100760 secure :-1 s=READY pgs=3087 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009900 tx=0x7fbfcc02bc30 comp rx=0 tx=0).stop 2024-03-24T12:43:54.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.718+0000 7fbfea822640 1 -- 172.21.15.6:0/3480114209 shutdown_connections 2024-03-24T12:43:54.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.718+0000 7fbfea822640 1 --2- 172.21.15.6:0/3480114209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfe41053a0 0x7fbfe410cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:54.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.718+0000 7fbfea822640 1 --2- 172.21.15.6:0/3480114209 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfe4100d30 0x7fbfe4104cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:54.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.718+0000 7fbfea822640 1 --2- 172.21.15.6:0/3480114209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfe4100380 0x7fbfe4100760 unknown :-1 s=CLOSED pgs=3087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:54.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.719+0000 7fbfea822640 1 -- 172.21.15.6:0/3480114209 >> 172.21.15.6:0/3480114209 conn(0x7fbfe4076190 msgr2=0x7fbfe40765a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:43:54.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.719+0000 7fbfea822640 1 -- 172.21.15.6:0/3480114209 shutdown_connections 2024-03-24T12:43:54.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.719+0000 7fbfea822640 1 -- 172.21.15.6:0/3480114209 wait complete. 2024-03-24T12:43:54.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.719+0000 7fbfea822640 1 Processor -- start 2024-03-24T12:43:54.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.719+0000 7fbfea822640 1 -- start start 2024-03-24T12:43:54.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.719+0000 7fbfea822640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfe4100380 0x7fbfe4197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:54.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.719+0000 7fbfea822640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfe41053a0 0x7fbfe4198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:54.717 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.719+0000 7fbfea822640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfe41985f0 0x7fbfe41a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:54.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.719+0000 7fbfea822640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe41a3240 con 0x7fbfe4100380 2024-03-24T12:43:54.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.719+0000 7fbfea822640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe41a33b0 con 0x7fbfe41053a0 2024-03-24T12:43:54.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.719+0000 7fbfea822640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe41a3570 con 0x7fbfe41985f0 2024-03-24T12:43:54.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.720+0000 7fbfe8dbf640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfe41985f0 0x7fbfe41a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:54.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.720+0000 7fbfe3fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfe4100380 0x7fbfe4197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:54.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.720+0000 7fbfe3fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfe4100380 0x7fbfe4197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:53260/0 (socket says 172.21.15.6:53260) 2024-03-24T12:43:54.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.720+0000 7fbfe8dbf640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfe41985f0 0x7fbfe41a2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:43486/0 (socket says 172.21.15.6:43486) 2024-03-24T12:43:54.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.720+0000 7fbfe3fff640 1 -- 172.21.15.6:0/4030177303 learned_addr learned my addr 172.21.15.6:0/4030177303 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:43:54.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.720+0000 7fbfe37fe640 1 --2- 172.21.15.6:0/4030177303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfe41053a0 0x7fbfe4198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:54.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.720+0000 7fbfe3fff640 1 -- 172.21.15.6:0/4030177303 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfe41985f0 msgr2=0x7fbfe41a2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:54.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.720+0000 7fbfe3fff640 1 --2- 172.21.15.6:0/4030177303 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfe41985f0 0x7fbfe41a2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:54.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.720+0000 7fbfe3fff640 1 -- 172.21.15.6:0/4030177303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfe41053a0 msgr2=0x7fbfe4198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:54.718 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.720+0000 7fbfe3fff640 1 --2- 172.21.15.6:0/4030177303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfe41053a0 0x7fbfe4198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:54.719 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.720+0000 7fbfe3fff640 1 -- 172.21.15.6:0/4030177303 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd4009490 con 0x7fbfe4100380 2024-03-24T12:43:54.719 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.720+0000 7fbfe8dbf640 1 --2- 172.21.15.6:0/4030177303 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfe41985f0 0x7fbfe41a2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:43:54.719 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.720+0000 7fbfe3fff640 1 --2- 172.21.15.6:0/4030177303 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfe4100380 0x7fbfe4197b20 secure :-1 s=READY pgs=3102 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc005940 tx=0x7fbfcc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:43:54.719 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.720+0000 7fbfe17fa640 1 -- 172.21.15.6:0/4030177303 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc0044e0 con 0x7fbfe4100380 2024-03-24T12:43:54.719 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.720+0000 7fbfe17fa640 1 -- 172.21.15.6:0/4030177303 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fbfcc003c80 con 0x7fbfe4100380 2024-03-24T12:43:54.719 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.720+0000 7fbfea822640 1 -- 172.21.15.6:0/4030177303 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfcc009560 con 0x7fbfe4100380 2024-03-24T12:43:54.719 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.720+0000 7fbfe17fa640 1 -- 172.21.15.6:0/4030177303 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc005510 con 0x7fbfe4100380 2024-03-24T12:43:54.719 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.720+0000 7fbfea822640 1 -- 172.21.15.6:0/4030177303 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe41a3b70 con 0x7fbfe4100380 2024-03-24T12:43:54.720 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.721+0000 7fbfea822640 1 -- 172.21.15.6:0/4030177303 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe41017a0 con 0x7fbfe4100380 2024-03-24T12:43:54.720 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.722+0000 7fbfe17fa640 1 -- 172.21.15.6:0/4030177303 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7fbfcc004d90 con 0x7fbfe4100380 2024-03-24T12:43:54.721 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.722+0000 7fbfe17fa640 1 --2- 172.21.15.6:0/4030177303 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fbfd006d5c0 0x7fbfd006fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:54.721 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.722+0000 7fbfe17fa640 1 -- 172.21.15.6:0/4030177303 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(779..779 src has 255..779) v4 ==== 9207+0+0 (secure 0 0 0) 0x7fbfcc0afe50 con 0x7fbfe4100380 2024-03-24T12:43:54.721 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.722+0000 7fbfe37fe640 1 --2- 172.21.15.6:0/4030177303 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fbfd006d5c0 0x7fbfd006fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:54.721 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.723+0000 7fbfe17fa640 1 -- 172.21.15.6:0/4030177303 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=780}) v3 -- 0x7fbfd0079260 con 0x7fbfe4100380 2024-03-24T12:43:54.721 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.723+0000 7fbfe37fe640 1 --2- 172.21.15.6:0/4030177303 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fbfd006d5c0 0x7fbfd006fa80 secure :-1 s=READY pgs=4116 cs=0 l=1 rev1=1 crypto rx=0x7fbfe406a350 tx=0x7fbfd4005f90 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:43:54.723 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.727+0000 7fbfe17fa640 1 -- 172.21.15.6:0/4030177303 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbfcc078d50 con 0x7fbfe4100380 2024-03-24T12:43:54.907 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:54.911+0000 7fbfea822640 1 -- 172.21.15.6:0/4030177303 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fbfe4069e50 con 0x7fbfe4100380 2024-03-24T12:43:54.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:54 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]': finished 2024-03-24T12:43:54.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:54 smithi090 ceph-mon[33646]: osdmap e778: 8 total, 8 up, 8 in 2024-03-24T12:43:54.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:54 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2193507516' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:43:54.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:54 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:43:54.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:54 smithi090 ceph-mon[33646]: pgmap v1725: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:43:54.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:54 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:43:54.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:54 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]': finished 2024-03-24T12:43:54.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:54 smithi006 ceph-mon[30108]: osdmap e778: 8 total, 8 up, 8 in 2024-03-24T12:43:54.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:54 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2193507516' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:43:54.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:54 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:43:54.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:54 smithi006 ceph-mon[30108]: pgmap v1725: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:43:54.958 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:54 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:43:54.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:54 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]': finished 2024-03-24T12:43:54.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:54 smithi006 ceph-mon[41734]: osdmap e778: 8 total, 8 up, 8 in 2024-03-24T12:43:54.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:54 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2193507516' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:43:54.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:54 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:43:54.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:54 smithi006 ceph-mon[41734]: pgmap v1725: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:43:54.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:54 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:43:55.606 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:55.610+0000 7fbfe17fa640 1 -- 172.21.15.6:0/4030177303 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 v780) v1 ==== 221+0+0 (secure 0 0 0) 0x7fbfcc0430d0 con 0x7fbfe4100380 2024-03-24T12:43:55.607 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:55.611+0000 7fbfe17fa640 1 -- 172.21.15.6:0/4030177303 <== mon.0 v2:172.21.15.6:3300/0 8 ==== osd_map(780..780 src has 255..780) v4 ==== 628+0+0 (secure 0 0 0) 0x7fbfcc09c060 con 0x7fbfe4100380 2024-03-24T12:43:55.607 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:55.611+0000 7fbfe17fa640 1 -- 172.21.15.6:0/4030177303 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=781}) v3 -- 0x7fbfd0079ca0 con 0x7fbfe4100380 2024-03-24T12:43:55.614 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:43:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:43:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:43:55.725 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:55.729+0000 7fbfea822640 1 -- 172.21.15.6:0/4030177303 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fbfe4100d30 con 0x7fbfe4100380 2024-03-24T12:43:55.924 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:55 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]': finished 2024-03-24T12:43:55.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:55 smithi090 ceph-mon[33646]: osdmap e779: 8 total, 8 up, 8 in 2024-03-24T12:43:55.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:55 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4030177303' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-24T12:43:55.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:55 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]': finished 2024-03-24T12:43:55.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:55 smithi006 ceph-mon[30108]: osdmap e779: 8 total, 8 up, 8 in 2024-03-24T12:43:55.957 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:55 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4030177303' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-24T12:43:55.957 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:55 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]': finished 2024-03-24T12:43:55.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:55 smithi006 ceph-mon[41734]: osdmap e779: 8 total, 8 up, 8 in 2024-03-24T12:43:55.958 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:55 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4030177303' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-24T12:43:56.161 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.165+0000 7fbfe17fa640 1 -- 172.21.15.6:0/4030177303 <== mon.0 v2:172.21.15.6:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 v781) v1 ==== 221+0+0 (secure 0 0 0) 0x7fbfcc07c410 con 0x7fbfe4100380 2024-03-24T12:43:56.161 INFO:tasks.workunit.client.0.smithi006.stderr:set-quota max_objects = 0 for pool 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 2024-03-24T12:43:56.161 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.165+0000 7fbfe17fa640 1 -- 172.21.15.6:0/4030177303 <== mon.0 v2:172.21.15.6:3300/0 10 ==== osd_map(781..781 src has 255..781) v4 ==== 628+0+0 (secure 0 0 0) 0x7fbfcc003df0 con 0x7fbfe4100380 2024-03-24T12:43:56.161 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.165+0000 7fbfe17fa640 1 -- 172.21.15.6:0/4030177303 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=782}) v3 -- 0x7fbfd007a230 con 0x7fbfe4100380 2024-03-24T12:43:56.166 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.170+0000 7fbfea822640 1 -- 172.21.15.6:0/4030177303 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fbfd006d5c0 msgr2=0x7fbfd006fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:56.166 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.170+0000 7fbfea822640 1 --2- 172.21.15.6:0/4030177303 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fbfd006d5c0 0x7fbfd006fa80 secure :-1 s=READY pgs=4116 cs=0 l=1 rev1=1 crypto rx=0x7fbfe406a350 tx=0x7fbfd4005f90 comp rx=0 tx=0).stop 2024-03-24T12:43:56.166 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.170+0000 7fbfea822640 1 -- 172.21.15.6:0/4030177303 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfe4100380 msgr2=0x7fbfe4197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:56.166 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.170+0000 7fbfea822640 1 --2- 172.21.15.6:0/4030177303 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfe4100380 0x7fbfe4197b20 secure :-1 s=READY pgs=3102 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc005940 tx=0x7fbfcc0042f0 comp rx=0 tx=0).stop 2024-03-24T12:43:56.166 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.170+0000 7fbfea822640 1 -- 172.21.15.6:0/4030177303 shutdown_connections 2024-03-24T12:43:56.166 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.170+0000 7fbfea822640 1 --2- 172.21.15.6:0/4030177303 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fbfd006d5c0 0x7fbfd006fa80 unknown :-1 s=CLOSED pgs=4116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:56.166 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.170+0000 7fbfea822640 1 --2- 172.21.15.6:0/4030177303 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbfe41985f0 0x7fbfe41a2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:56.167 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.170+0000 7fbfea822640 1 --2- 172.21.15.6:0/4030177303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfe41053a0 0x7fbfe4198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:56.167 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.170+0000 7fbfea822640 1 --2- 172.21.15.6:0/4030177303 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfe4100380 0x7fbfe4197b20 unknown :-1 s=CLOSED pgs=3102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:56.167 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.170+0000 7fbfea822640 1 -- 172.21.15.6:0/4030177303 >> 172.21.15.6:0/4030177303 conn(0x7fbfe4076190 msgr2=0x7fbfe40fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:43:56.167 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.170+0000 7fbfea822640 1 -- 172.21.15.6:0/4030177303 shutdown_connections 2024-03-24T12:43:56.167 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.170+0000 7fbfea822640 1 -- 172.21.15.6:0/4030177303 wait complete. 2024-03-24T12:43:56.178 INFO:tasks.workunit.client.0.smithi006.stderr:+ wait 158757 2024-03-24T12:43:56.179 INFO:tasks.workunit.client.0.smithi006.stderr:+ '[' 0 -ne 0 ']' 2024-03-24T12:43:56.179 INFO:tasks.workunit.client.0.smithi006.stderr:+ true 2024-03-24T12:43:56.179 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 put three /etc/passwd 2024-03-24T12:43:56.215 INFO:tasks.workunit.client.0.smithi006.stderr:++ uuidgen 2024-03-24T12:43:56.216 INFO:tasks.workunit.client.0.smithi006.stderr:+ pp=f99c589a-562e-40f2-a1ef-11b16b6c29e6 2024-03-24T12:43:56.216 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph osd pool create f99c589a-562e-40f2-a1ef-11b16b6c29e6 12 2024-03-24T12:43:56.304 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.306+0000 7fb75cbbc640 1 -- 172.21.15.6:0/3305622074 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7581017b0 msgr2=0x7fb75810cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:56.304 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.306+0000 7fb75cbbc640 1 --2- 172.21.15.6:0/3305622074 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7581017b0 0x7fb75810cf30 secure :-1 s=READY pgs=3105 cs=0 l=1 rev1=1 crypto rx=0x7fb750009cd0 tx=0x7fb75002c0f0 comp rx=0 tx=0).stop 2024-03-24T12:43:56.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.306+0000 7fb75cbbc640 1 -- 172.21.15.6:0/3305622074 shutdown_connections 2024-03-24T12:43:56.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.306+0000 7fb75cbbc640 1 --2- 172.21.15.6:0/3305622074 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7581017b0 0x7fb75810cf30 unknown :-1 s=CLOSED pgs=3105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:56.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.306+0000 7fb75cbbc640 1 --2- 172.21.15.6:0/3305622074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb758100df0 0x7fb758101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:56.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.306+0000 7fb75cbbc640 1 --2- 172.21.15.6:0/3305622074 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb7581061f0 0x7fb7581065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:56.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.306+0000 7fb75cbbc640 1 -- 172.21.15.6:0/3305622074 >> 172.21.15.6:0/3305622074 conn(0x7fb7580753d0 msgr2=0x7fb7580757e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:43:56.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.306+0000 7fb75cbbc640 1 -- 172.21.15.6:0/3305622074 shutdown_connections 2024-03-24T12:43:56.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.306+0000 7fb75cbbc640 1 -- 172.21.15.6:0/3305622074 wait complete. 2024-03-24T12:43:56.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.307+0000 7fb75cbbc640 1 Processor -- start 2024-03-24T12:43:56.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.307+0000 7fb75cbbc640 1 -- start start 2024-03-24T12:43:56.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.307+0000 7fb75cbbc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb758100df0 0x7fb758197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:56.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.307+0000 7fb75cbbc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb7581017b0 0x7fb758198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:56.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.307+0000 7fb75cbbc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7581061f0 0x7fb7581a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:56.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.307+0000 7fb75cbbc640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb758198880 con 0x7fb7581061f0 2024-03-24T12:43:56.305 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.307+0000 7fb75cbbc640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7581989f0 con 0x7fb758100df0 2024-03-24T12:43:56.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.307+0000 7fb75cbbc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7581a3160 con 0x7fb7581017b0 2024-03-24T12:43:56.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.307+0000 7fb756d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7581061f0 0x7fb7581a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:56.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.307+0000 7fb756d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7581061f0 0x7fb7581a2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46090/0 (socket says 172.21.15.6:46090) 2024-03-24T12:43:56.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.307+0000 7fb74ffff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb7581017b0 0x7fb758198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:56.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.307+0000 7fb756d9d640 1 -- 172.21.15.6:0/2496656475 learned_addr learned my addr 172.21.15.6:0/2496656475 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:43:56.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.308+0000 7fb75659c640 1 --2- 172.21.15.6:0/2496656475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb758100df0 0x7fb758197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:56.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.308+0000 7fb74ffff640 1 -- 172.21.15.6:0/2496656475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb758100df0 msgr2=0x7fb758197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:56.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.308+0000 7fb74ffff640 1 --2- 172.21.15.6:0/2496656475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb758100df0 0x7fb758197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:56.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.308+0000 7fb74ffff640 1 -- 172.21.15.6:0/2496656475 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7581061f0 msgr2=0x7fb7581a2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:56.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.308+0000 7fb74ffff640 1 --2- 172.21.15.6:0/2496656475 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7581061f0 0x7fb7581a2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:56.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.308+0000 7fb74ffff640 1 -- 172.21.15.6:0/2496656475 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb748009490 con 0x7fb7581017b0 2024-03-24T12:43:56.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.308+0000 7fb756d9d640 1 --2- 172.21.15.6:0/2496656475 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7581061f0 0x7fb7581a2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:43:56.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.308+0000 7fb74ffff640 1 --2- 172.21.15.6:0/2496656475 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb7581017b0 0x7fb758198090 secure :-1 s=READY pgs=3089 cs=0 l=1 rev1=1 crypto rx=0x7fb74000c970 tx=0x7fb74000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:43:56.306 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.308+0000 7fb74f7fe640 1 -- 172.21.15.6:0/2496656475 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb740007bb0 con 0x7fb7581017b0 2024-03-24T12:43:56.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.308+0000 7fb74f7fe640 1 -- 172.21.15.6:0/2496656475 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fb740007d10 con 0x7fb7581017b0 2024-03-24T12:43:56.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.308+0000 7fb74f7fe640 1 -- 172.21.15.6:0/2496656475 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb740004cd0 con 0x7fb7581017b0 2024-03-24T12:43:56.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.308+0000 7fb75cbbc640 1 -- 172.21.15.6:0/2496656475 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb750009900 con 0x7fb7581017b0 2024-03-24T12:43:56.307 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.308+0000 7fb75cbbc640 1 -- 172.21.15.6:0/2496656475 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7581a37c0 con 0x7fb7581017b0 2024-03-24T12:43:56.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.309+0000 7fb75cbbc640 1 -- 172.21.15.6:0/2496656475 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb758067940 con 0x7fb7581017b0 2024-03-24T12:43:56.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.310+0000 7fb74f7fe640 1 -- 172.21.15.6:0/2496656475 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7fb740004e30 con 0x7fb7581017b0 2024-03-24T12:43:56.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.310+0000 7fb74f7fe640 1 --2- 172.21.15.6:0/2496656475 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fb74406d5d0 0x7fb74406fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:56.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.311+0000 7fb74f7fe640 1 -- 172.21.15.6:0/2496656475 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(781..781 src has 255..781) v4 ==== 9207+0+0 (secure 0 0 0) 0x7fb74008da80 con 0x7fb7581017b0 2024-03-24T12:43:56.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.311+0000 7fb75659c640 1 --2- 172.21.15.6:0/2496656475 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fb74406d5d0 0x7fb74406fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:56.308 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.311+0000 7fb75659c640 1 --2- 172.21.15.6:0/2496656475 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fb74406d5d0 0x7fb74406fa90 secure :-1 s=READY pgs=4118 cs=0 l=1 rev1=1 crypto rx=0x7fb748009c70 tx=0x7fb748005f90 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:43:56.314 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.319+0000 7fb74f7fe640 1 -- 172.21.15.6:0/2496656475 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb740056a30 con 0x7fb7581017b0 2024-03-24T12:43:56.498 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:56.502+0000 7fb75cbbc640 1 -- 172.21.15.6:0/2496656475 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pg_num": 12} v 0) v1 -- 0x7fb75810f660 con 0x7fb7581017b0 2024-03-24T12:43:57.173 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.177+0000 7fb74f7fe640 1 -- 172.21.15.6:0/2496656475 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pg_num": 12}]=0 pool 'f99c589a-562e-40f2-a1ef-11b16b6c29e6' created v782) v1 ==== 176+0+0 (secure 0 0 0) 0x7fb74001d0c0 con 0x7fb7581017b0 2024-03-24T12:43:57.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:56 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4030177303' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]': finished 2024-03-24T12:43:57.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:56 smithi090 ceph-mon[33646]: osdmap e780: 8 total, 8 up, 8 in 2024-03-24T12:43:57.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:56 smithi090 ceph-mon[33646]: pgmap v1728: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:43:57.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:56 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4030177303' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-24T12:43:57.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:56 smithi090 ceph-mon[33646]: pool '54e008b8-0c65-4b2f-9ae3-d0f6346347d3' no longer out of quota; removing NO_QUOTA flag 2024-03-24T12:43:57.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:56 smithi090 ceph-mon[33646]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-24T12:43:57.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:56 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/4030177303' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]': finished 2024-03-24T12:43:57.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:56 smithi090 ceph-mon[33646]: osdmap e781: 8 total, 8 up, 8 in 2024-03-24T12:43:57.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:56 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4030177303' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]': finished 2024-03-24T12:43:57.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:56 smithi006 ceph-mon[30108]: osdmap e780: 8 total, 8 up, 8 in 2024-03-24T12:43:57.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:56 smithi006 ceph-mon[30108]: pgmap v1728: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:43:57.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:56 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4030177303' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-24T12:43:57.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:56 smithi006 ceph-mon[30108]: pool '54e008b8-0c65-4b2f-9ae3-d0f6346347d3' no longer out of quota; removing NO_QUOTA flag 2024-03-24T12:43:57.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:56 smithi006 ceph-mon[30108]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-24T12:43:57.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:56 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/4030177303' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]': finished 2024-03-24T12:43:57.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:56 smithi006 ceph-mon[30108]: osdmap e781: 8 total, 8 up, 8 in 2024-03-24T12:43:57.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:56 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4030177303' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]': finished 2024-03-24T12:43:57.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:56 smithi006 ceph-mon[41734]: osdmap e780: 8 total, 8 up, 8 in 2024-03-24T12:43:57.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:56 smithi006 ceph-mon[41734]: pgmap v1728: 176 pgs: 176 active+clean; 721 KiB data, 870 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:43:57.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:56 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4030177303' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-24T12:43:57.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:56 smithi006 ceph-mon[41734]: pool '54e008b8-0c65-4b2f-9ae3-d0f6346347d3' no longer out of quota; removing NO_QUOTA flag 2024-03-24T12:43:57.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:56 smithi006 ceph-mon[41734]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-24T12:43:57.209 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:56 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/4030177303' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]': finished 2024-03-24T12:43:57.209 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:56 smithi006 ceph-mon[41734]: osdmap e781: 8 total, 8 up, 8 in 2024-03-24T12:43:57.284 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.289+0000 7fb75cbbc640 1 -- 172.21.15.6:0/2496656475 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pg_num": 12} v 0) v1 -- 0x7fb75810f7f0 con 0x7fb7581017b0 2024-03-24T12:43:57.285 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.290+0000 7fb74f7fe640 1 -- 172.21.15.6:0/2496656475 <== mon.2 v2:172.21.15.6:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pg_num": 12}]=0 pool 'f99c589a-562e-40f2-a1ef-11b16b6c29e6' already exists v782) v1 ==== 183+0+0 (secure 0 0 0) 0x7fb74005a0f0 con 0x7fb7581017b0 2024-03-24T12:43:57.285 INFO:tasks.workunit.client.0.smithi006.stderr:pool 'f99c589a-562e-40f2-a1ef-11b16b6c29e6' already exists 2024-03-24T12:43:57.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.292+0000 7fb75cbbc640 1 -- 172.21.15.6:0/2496656475 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fb74406d5d0 msgr2=0x7fb74406fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:57.287 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.292+0000 7fb75cbbc640 1 --2- 172.21.15.6:0/2496656475 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fb74406d5d0 0x7fb74406fa90 secure :-1 s=READY pgs=4118 cs=0 l=1 rev1=1 crypto rx=0x7fb748009c70 tx=0x7fb748005f90 comp rx=0 tx=0).stop 2024-03-24T12:43:57.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.292+0000 7fb75cbbc640 1 -- 172.21.15.6:0/2496656475 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb7581017b0 msgr2=0x7fb758198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:57.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.292+0000 7fb75cbbc640 1 --2- 172.21.15.6:0/2496656475 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb7581017b0 0x7fb758198090 secure :-1 s=READY pgs=3089 cs=0 l=1 rev1=1 crypto rx=0x7fb74000c970 tx=0x7fb74000ce30 comp rx=0 tx=0).stop 2024-03-24T12:43:57.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.292+0000 7fb75cbbc640 1 -- 172.21.15.6:0/2496656475 shutdown_connections 2024-03-24T12:43:57.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.292+0000 7fb75cbbc640 1 --2- 172.21.15.6:0/2496656475 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fb74406d5d0 0x7fb74406fa90 unknown :-1 s=CLOSED pgs=4118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:57.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.292+0000 7fb75cbbc640 1 --2- 172.21.15.6:0/2496656475 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7581061f0 0x7fb7581a2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:57.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.292+0000 7fb75cbbc640 1 --2- 172.21.15.6:0/2496656475 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb7581017b0 0x7fb758198090 unknown :-1 s=CLOSED pgs=3089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:57.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.292+0000 7fb75cbbc640 1 --2- 172.21.15.6:0/2496656475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb758100df0 0x7fb758197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:57.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.292+0000 7fb75cbbc640 1 -- 172.21.15.6:0/2496656475 >> 172.21.15.6:0/2496656475 conn(0x7fb7580753d0 msgr2=0x7fb758103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:43:57.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.292+0000 7fb75cbbc640 1 -- 172.21.15.6:0/2496656475 shutdown_connections 2024-03-24T12:43:57.288 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.292+0000 7fb75cbbc640 1 -- 172.21.15.6:0/2496656475 wait complete. 2024-03-24T12:43:57.296 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph osd pool application enable f99c589a-562e-40f2-a1ef-11b16b6c29e6 rados 2024-03-24T12:43:57.382 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.385+0000 7fc1eb239640 1 -- 172.21.15.6:0/1960312733 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc1e4101e30 msgr2=0x7fc1e4105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:57.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.385+0000 7fc1eb239640 1 --2- 172.21.15.6:0/1960312733 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc1e4101e30 0x7fc1e4105e70 secure :-1 s=READY pgs=3090 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0009900 tx=0x7fc1d002bc30 comp rx=0 tx=0).stop 2024-03-24T12:43:57.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.385+0000 7fc1eb239640 1 -- 172.21.15.6:0/1960312733 shutdown_connections 2024-03-24T12:43:57.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.385+0000 7fc1eb239640 1 --2- 172.21.15.6:0/1960312733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1e4106550 0x7fc1e410e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:57.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.385+0000 7fc1eb239640 1 --2- 172.21.15.6:0/1960312733 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc1e4101e30 0x7fc1e4105e70 unknown :-1 s=CLOSED pgs=3090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:57.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.385+0000 7fc1eb239640 1 --2- 172.21.15.6:0/1960312733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1e4101480 0x7fc1e4101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:57.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.385+0000 7fc1eb239640 1 -- 172.21.15.6:0/1960312733 >> 172.21.15.6:0/1960312733 conn(0x7fc1e40fccc0 msgr2=0x7fc1e40ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:43:57.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.385+0000 7fc1eb239640 1 -- 172.21.15.6:0/1960312733 shutdown_connections 2024-03-24T12:43:57.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.385+0000 7fc1eb239640 1 -- 172.21.15.6:0/1960312733 wait complete. 2024-03-24T12:43:57.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.385+0000 7fc1eb239640 1 Processor -- start 2024-03-24T12:43:57.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.385+0000 7fc1eb239640 1 -- start start 2024-03-24T12:43:57.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.385+0000 7fc1eb239640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc1e4101480 0x7fc1e4073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:57.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.385+0000 7fc1eb239640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1e4106550 0x7fc1e40739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:57.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.385+0000 7fc1eb239640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1e4074010 0x7fc1e41a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:57.383 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.386+0000 7fc1eb239640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e41a57e0 con 0x7fc1e4106550 2024-03-24T12:43:57.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.386+0000 7fc1eb239640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e41a5950 con 0x7fc1e4074010 2024-03-24T12:43:57.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.386+0000 7fc1eb239640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e41a5b10 con 0x7fc1e4101480 2024-03-24T12:43:57.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.386+0000 7fc1e8fd5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc1e4101480 0x7fc1e4073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:57.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.386+0000 7fc1e3fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1e4106550 0x7fc1e40739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:57.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.386+0000 7fc1e8fd5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc1e4101480 0x7fc1e4073480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:38598/0 (socket says 172.21.15.6:38598) 2024-03-24T12:43:57.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.386+0000 7fc1e3fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1e4106550 0x7fc1e40739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46102/0 (socket says 172.21.15.6:46102) 2024-03-24T12:43:57.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.386+0000 7fc1e8fd5640 1 -- 172.21.15.6:0/193845149 learned_addr learned my addr 172.21.15.6:0/193845149 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:43:57.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.386+0000 7fc1e8fd5640 1 -- 172.21.15.6:0/193845149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1e4074010 msgr2=0x7fc1e41a5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:57.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.386+0000 7fc1e8fd5640 1 --2- 172.21.15.6:0/193845149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1e4074010 0x7fc1e41a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:57.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.386+0000 7fc1e8fd5640 1 -- 172.21.15.6:0/193845149 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1e4106550 msgr2=0x7fc1e40739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:57.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.386+0000 7fc1e8fd5640 1 --2- 172.21.15.6:0/193845149 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1e4106550 0x7fc1e40739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:57.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.386+0000 7fc1e8fd5640 1 -- 172.21.15.6:0/193845149 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1dc009840 con 0x7fc1e4101480 2024-03-24T12:43:57.384 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.386+0000 7fc1e8fd5640 1 --2- 172.21.15.6:0/193845149 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc1e4101480 0x7fc1e4073480 secure :-1 s=READY pgs=3091 cs=0 l=1 rev1=1 crypto rx=0x7fc1d800b4a0 tx=0x7fc1d800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:43:57.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.386+0000 7fc1e1ffb640 1 -- 172.21.15.6:0/193845149 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8010610 con 0x7fc1e4101480 2024-03-24T12:43:57.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.386+0000 7fc1e1ffb640 1 -- 172.21.15.6:0/193845149 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fc1d8010bf0 con 0x7fc1e4101480 2024-03-24T12:43:57.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.386+0000 7fc1eb239640 1 -- 172.21.15.6:0/193845149 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d0009560 con 0x7fc1e4101480 2024-03-24T12:43:57.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.386+0000 7fc1e1ffb640 1 -- 172.21.15.6:0/193845149 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8017bc0 con 0x7fc1e4101480 2024-03-24T12:43:57.385 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.386+0000 7fc1eb239640 1 -- 172.21.15.6:0/193845149 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e41a6140 con 0x7fc1e4101480 2024-03-24T12:43:57.386 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.387+0000 7fc1eb239640 1 -- 172.21.15.6:0/193845149 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e4100950 con 0x7fc1e4101480 2024-03-24T12:43:57.386 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.388+0000 7fc1e1ffb640 1 -- 172.21.15.6:0/193845149 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7fc1d8010770 con 0x7fc1e4101480 2024-03-24T12:43:57.386 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.388+0000 7fc1e1ffb640 1 --2- 172.21.15.6:0/193845149 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fc1d406d5d0 0x7fc1d406fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:57.386 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.388+0000 7fc1e1ffb640 1 -- 172.21.15.6:0/193845149 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(782..782 src has 255..782) v4 ==== 9582+0+0 (secure 0 0 0) 0x7fc1d808d1e0 con 0x7fc1e4101480 2024-03-24T12:43:57.386 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.388+0000 7fc1e3fff640 1 --2- 172.21.15.6:0/193845149 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fc1d406d5d0 0x7fc1d406fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:57.386 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.389+0000 7fc1e3fff640 1 --2- 172.21.15.6:0/193845149 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fc1d406d5d0 0x7fc1d406fa90 secure :-1 s=READY pgs=4119 cs=0 l=1 rev1=1 crypto rx=0x7fc1d002c240 tx=0x7fc1d0005a20 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:43:57.388 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.393+0000 7fc1e1ffb640 1 -- 172.21.15.6:0/193845149 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc1d8056080 con 0x7fc1e4101480 2024-03-24T12:43:57.570 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:57.574+0000 7fc1eb239640 1 -- 172.21.15.6:0/193845149 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "app": "rados"} v 0) v1 -- 0x7fc1e404efc0 con 0x7fc1e4101480 2024-03-24T12:43:58.076 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:57 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2496656475' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pg_num": 12}]: dispatch 2024-03-24T12:43:58.076 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:57 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pg_num": 12}]: dispatch 2024-03-24T12:43:58.076 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:57 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pg_num": 12}]': finished 2024-03-24T12:43:58.076 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:57 smithi090 ceph-mon[33646]: osdmap e782: 8 total, 8 up, 8 in 2024-03-24T12:43:58.076 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:57 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2496656475' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pg_num": 12}]: dispatch 2024-03-24T12:43:58.076 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:57 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pg_num": 12}]: dispatch 2024-03-24T12:43:58.076 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:57 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/193845149' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "app": "rados"}]: dispatch 2024-03-24T12:43:58.077 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:57 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "app": "rados"}]: dispatch 2024-03-24T12:43:58.178 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:58.182+0000 7fc1e1ffb640 1 -- 172.21.15.6:0/193845149 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "app": "rados"}]=0 enabled application 'rados' on pool 'f99c589a-562e-40f2-a1ef-11b16b6c29e6' v783) v1 ==== 213+0+0 (secure 0 0 0) 0x7fc1d8059740 con 0x7fc1e4101480 2024-03-24T12:43:58.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:57 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2496656475' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pg_num": 12}]: dispatch 2024-03-24T12:43:58.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:57 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pg_num": 12}]: dispatch 2024-03-24T12:43:58.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:57 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pg_num": 12}]': finished 2024-03-24T12:43:58.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:57 smithi006 ceph-mon[30108]: osdmap e782: 8 total, 8 up, 8 in 2024-03-24T12:43:58.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:57 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2496656475' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pg_num": 12}]: dispatch 2024-03-24T12:43:58.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:57 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pg_num": 12}]: dispatch 2024-03-24T12:43:58.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:57 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/193845149' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "app": "rados"}]: dispatch 2024-03-24T12:43:58.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:57 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "app": "rados"}]: dispatch 2024-03-24T12:43:58.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:57 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2496656475' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pg_num": 12}]: dispatch 2024-03-24T12:43:58.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:57 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pg_num": 12}]: dispatch 2024-03-24T12:43:58.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:57 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pg_num": 12}]': finished 2024-03-24T12:43:58.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:57 smithi006 ceph-mon[41734]: osdmap e782: 8 total, 8 up, 8 in 2024-03-24T12:43:58.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:57 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2496656475' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pg_num": 12}]: dispatch 2024-03-24T12:43:58.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:57 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pg_num": 12}]: dispatch 2024-03-24T12:43:58.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:57 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/193845149' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "app": "rados"}]: dispatch 2024-03-24T12:43:58.209 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:57 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "app": "rados"}]: dispatch 2024-03-24T12:43:58.299 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:58.303+0000 7fc1eb239640 1 -- 172.21.15.6:0/193845149 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "app": "rados"} v 0) v1 -- 0x7fc1e4101e30 con 0x7fc1e4101480 2024-03-24T12:43:59.089 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:58 smithi090 ceph-mon[33646]: pgmap v1731: 188 pgs: 12 unknown, 176 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-03-24T12:43:59.089 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:58 smithi090 ceph-mon[33646]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:43:59.089 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:58 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "app": "rados"}]': finished 2024-03-24T12:43:59.089 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:58 smithi090 ceph-mon[33646]: osdmap e783: 8 total, 8 up, 8 in 2024-03-24T12:43:59.089 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:58 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/193845149' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "app": "rados"}]: dispatch 2024-03-24T12:43:59.089 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:43:58 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "app": "rados"}]: dispatch 2024-03-24T12:43:59.182 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.186+0000 7fc1e1ffb640 1 -- 172.21.15.6:0/193845149 <== mon.2 v2:172.21.15.6:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "app": "rados"}]=0 enabled application 'rados' on pool 'f99c589a-562e-40f2-a1ef-11b16b6c29e6' v784) v1 ==== 213+0+0 (secure 0 0 0) 0x7fc1d80140d0 con 0x7fc1e4101480 2024-03-24T12:43:59.183 INFO:tasks.workunit.client.0.smithi006.stderr:enabled application 'rados' on pool 'f99c589a-562e-40f2-a1ef-11b16b6c29e6' 2024-03-24T12:43:59.187 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.190+0000 7fc1eb239640 1 -- 172.21.15.6:0/193845149 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fc1d406d5d0 msgr2=0x7fc1d406fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:59.187 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.190+0000 7fc1eb239640 1 --2- 172.21.15.6:0/193845149 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fc1d406d5d0 0x7fc1d406fa90 secure :-1 s=READY pgs=4119 cs=0 l=1 rev1=1 crypto rx=0x7fc1d002c240 tx=0x7fc1d0005a20 comp rx=0 tx=0).stop 2024-03-24T12:43:59.187 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.190+0000 7fc1eb239640 1 -- 172.21.15.6:0/193845149 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc1e4101480 msgr2=0x7fc1e4073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:59.187 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.190+0000 7fc1eb239640 1 --2- 172.21.15.6:0/193845149 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc1e4101480 0x7fc1e4073480 secure :-1 s=READY pgs=3091 cs=0 l=1 rev1=1 crypto rx=0x7fc1d800b4a0 tx=0x7fc1d800b960 comp rx=0 tx=0).stop 2024-03-24T12:43:59.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.190+0000 7fc1eb239640 1 -- 172.21.15.6:0/193845149 shutdown_connections 2024-03-24T12:43:59.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.190+0000 7fc1eb239640 1 --2- 172.21.15.6:0/193845149 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fc1d406d5d0 0x7fc1d406fa90 unknown :-1 s=CLOSED pgs=4119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:59.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.190+0000 7fc1eb239640 1 --2- 172.21.15.6:0/193845149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1e4074010 0x7fc1e41a5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:59.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.190+0000 7fc1eb239640 1 --2- 172.21.15.6:0/193845149 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1e4106550 0x7fc1e40739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:59.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.190+0000 7fc1eb239640 1 --2- 172.21.15.6:0/193845149 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc1e4101480 0x7fc1e4073480 unknown :-1 s=CLOSED pgs=3091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:59.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.190+0000 7fc1eb239640 1 -- 172.21.15.6:0/193845149 >> 172.21.15.6:0/193845149 conn(0x7fc1e40fccc0 msgr2=0x7fc1e40fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:43:59.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.190+0000 7fc1eb239640 1 -- 172.21.15.6:0/193845149 shutdown_connections 2024-03-24T12:43:59.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.190+0000 7fc1eb239640 1 -- 172.21.15.6:0/193845149 wait complete. 2024-03-24T12:43:59.198 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph osd pool set-quota f99c589a-562e-40f2-a1ef-11b16b6c29e6 max_objects 10 2024-03-24T12:43:59.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:58 smithi006 ceph-mon[30108]: pgmap v1731: 188 pgs: 12 unknown, 176 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-03-24T12:43:59.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:58 smithi006 ceph-mon[30108]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:43:59.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:58 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "app": "rados"}]': finished 2024-03-24T12:43:59.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:58 smithi006 ceph-mon[30108]: osdmap e783: 8 total, 8 up, 8 in 2024-03-24T12:43:59.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:58 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/193845149' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "app": "rados"}]: dispatch 2024-03-24T12:43:59.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:43:58 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "app": "rados"}]: dispatch 2024-03-24T12:43:59.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:58 smithi006 ceph-mon[41734]: pgmap v1731: 188 pgs: 12 unknown, 176 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-03-24T12:43:59.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:58 smithi006 ceph-mon[41734]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:43:59.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:58 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "app": "rados"}]': finished 2024-03-24T12:43:59.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:58 smithi006 ceph-mon[41734]: osdmap e783: 8 total, 8 up, 8 in 2024-03-24T12:43:59.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:58 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/193845149' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "app": "rados"}]: dispatch 2024-03-24T12:43:59.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:43:58 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "app": "rados"}]: dispatch 2024-03-24T12:43:59.291 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.293+0000 7f9466f56640 1 -- 172.21.15.6:0/2566611795 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9460100300 msgr2=0x7f94601006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:59.291 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.293+0000 7f9466f56640 1 --2- 172.21.15.6:0/2566611795 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9460100300 0x7f94601006e0 secure :-1 s=READY pgs=3106 cs=0 l=1 rev1=1 crypto rx=0x7f945c009930 tx=0x7f945c02bc30 comp rx=0 tx=0).stop 2024-03-24T12:43:59.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.293+0000 7f9466f56640 1 -- 172.21.15.6:0/2566611795 shutdown_connections 2024-03-24T12:43:59.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.293+0000 7f9466f56640 1 --2- 172.21.15.6:0/2566611795 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94601053d0 0x7f946010cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:59.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.293+0000 7f9466f56640 1 --2- 172.21.15.6:0/2566611795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9460100cb0 0x7f9460104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:59.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.293+0000 7f9466f56640 1 --2- 172.21.15.6:0/2566611795 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9460100300 0x7f94601006e0 unknown :-1 s=CLOSED pgs=3106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:59.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.293+0000 7f9466f56640 1 -- 172.21.15.6:0/2566611795 >> 172.21.15.6:0/2566611795 conn(0x7f94600771d0 msgr2=0x7f94600775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:43:59.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.293+0000 7f9466f56640 1 -- 172.21.15.6:0/2566611795 shutdown_connections 2024-03-24T12:43:59.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.293+0000 7f9466f56640 1 -- 172.21.15.6:0/2566611795 wait complete. 2024-03-24T12:43:59.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.294+0000 7f9466f56640 1 Processor -- start 2024-03-24T12:43:59.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.294+0000 7f9466f56640 1 -- start start 2024-03-24T12:43:59.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.294+0000 7f9466f56640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9460100300 0x7f9460196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:59.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.294+0000 7f9466f56640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94601053d0 0x7f9460197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:59.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.294+0000 7f9466f56640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f946019b710 0x7f946019bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:59.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.294+0000 7f9466f56640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946019c100 con 0x7f9460100300 2024-03-24T12:43:59.292 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.294+0000 7f9466f56640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946019c270 con 0x7f946019b710 2024-03-24T12:43:59.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.294+0000 7f9466f56640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946019c3e0 con 0x7f94601053d0 2024-03-24T12:43:59.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.294+0000 7f9464cf2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9460100300 0x7f9460196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:59.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.294+0000 7f9457fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94601053d0 0x7f9460197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:59.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.294+0000 7f9464cf2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9460100300 0x7f9460196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46120/0 (socket says 172.21.15.6:46120) 2024-03-24T12:43:59.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.294+0000 7f9464cf2640 1 -- 172.21.15.6:0/2145097865 learned_addr learned my addr 172.21.15.6:0/2145097865 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:43:59.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.294+0000 7f94654f3640 1 --2- 172.21.15.6:0/2145097865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f946019b710 0x7f946019bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:59.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.294+0000 7f9457fff640 1 -- 172.21.15.6:0/2145097865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f946019b710 msgr2=0x7f946019bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:59.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.294+0000 7f9457fff640 1 --2- 172.21.15.6:0/2145097865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f946019b710 0x7f946019bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:59.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.294+0000 7f9457fff640 1 -- 172.21.15.6:0/2145097865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9460100300 msgr2=0x7f9460196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:43:59.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.294+0000 7f9457fff640 1 --2- 172.21.15.6:0/2145097865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9460100300 0x7f9460196e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:43:59.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.295+0000 7f9457fff640 1 -- 172.21.15.6:0/2145097865 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945c009560 con 0x7f94601053d0 2024-03-24T12:43:59.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.295+0000 7f9464cf2640 1 --2- 172.21.15.6:0/2145097865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9460100300 0x7f9460196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:43:59.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.295+0000 7f9457fff640 1 --2- 172.21.15.6:0/2145097865 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94601053d0 0x7f9460197390 secure :-1 s=READY pgs=3092 cs=0 l=1 rev1=1 crypto rx=0x7f945000e9c0 tx=0x7f945000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:43:59.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.295+0000 7f9455ffb640 1 -- 172.21.15.6:0/2145097865 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945000ccc0 con 0x7f94601053d0 2024-03-24T12:43:59.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.295+0000 7f9455ffb640 1 -- 172.21.15.6:0/2145097865 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f945000ce20 con 0x7f94601053d0 2024-03-24T12:43:59.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.295+0000 7f9466f56640 1 -- 172.21.15.6:0/2145097865 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94601a39c0 con 0x7f94601053d0 2024-03-24T12:43:59.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.295+0000 7f9455ffb640 1 -- 172.21.15.6:0/2145097865 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9450018680 con 0x7f94601053d0 2024-03-24T12:43:59.294 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.295+0000 7f9466f56640 1 -- 172.21.15.6:0/2145097865 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94601a3e70 con 0x7f94601053d0 2024-03-24T12:43:59.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.296+0000 7f9466f56640 1 -- 172.21.15.6:0/2145097865 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946004ef50 con 0x7f94601053d0 2024-03-24T12:43:59.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.297+0000 7f9455ffb640 1 -- 172.21.15.6:0/2145097865 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7f9450010430 con 0x7f94601053d0 2024-03-24T12:43:59.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.297+0000 7f9455ffb640 1 --2- 172.21.15.6:0/2145097865 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f944806d5c0 0x7f944806fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:43:59.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.297+0000 7f9464cf2640 1 --2- 172.21.15.6:0/2145097865 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f944806d5c0 0x7f944806fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:43:59.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.297+0000 7f9455ffb640 1 -- 172.21.15.6:0/2145097865 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(784..784 src has 255..784) v4 ==== 9595+0+0 (secure 0 0 0) 0x7f9450014070 con 0x7f94601053d0 2024-03-24T12:43:59.296 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.298+0000 7f9464cf2640 1 --2- 172.21.15.6:0/2145097865 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f944806d5c0 0x7f944806fa80 secure :-1 s=READY pgs=4120 cs=0 l=1 rev1=1 crypto rx=0x7f945c000c00 tx=0x7f945c0058d0 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:43:59.297 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.302+0000 7f9455ffb640 1 -- 172.21.15.6:0/2145097865 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9450052d20 con 0x7f94601053d0 2024-03-24T12:43:59.481 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:43:59.485+0000 7f9466f56640 1 -- 172.21.15.6:0/2145097865 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f9460100cb0 con 0x7f94601053d0 2024-03-24T12:44:00.186 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:44:00.191+0000 7f9455ffb640 1 -- 172.21.15.6:0/2145097865 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool f99c589a-562e-40f2-a1ef-11b16b6c29e6 v785) v1 ==== 223+0+0 (secure 0 0 0) 0x7f9450056d60 con 0x7f94601053d0 2024-03-24T12:44:00.300 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:44:00.304+0000 7f9466f56640 1 -- 172.21.15.6:0/2145097865 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f9460100e40 con 0x7f94601053d0 2024-03-24T12:44:00.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:00 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "app": "rados"}]': finished 2024-03-24T12:44:00.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:00 smithi006 ceph-mon[30108]: osdmap e784: 8 total, 8 up, 8 in 2024-03-24T12:44:00.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:00 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2145097865' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:44:00.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:00 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:44:00.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:00 smithi006 ceph-mon[30108]: pgmap v1734: 188 pgs: 12 unknown, 176 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-03-24T12:44:00.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:00 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "app": "rados"}]': finished 2024-03-24T12:44:00.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:00 smithi006 ceph-mon[41734]: osdmap e784: 8 total, 8 up, 8 in 2024-03-24T12:44:00.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:00 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2145097865' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:44:00.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:00 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:44:00.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:00 smithi006 ceph-mon[41734]: pgmap v1734: 188 pgs: 12 unknown, 176 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-03-24T12:44:00.674 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:00 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "app": "rados"}]': finished 2024-03-24T12:44:00.675 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:00 smithi090 ceph-mon[33646]: osdmap e784: 8 total, 8 up, 8 in 2024-03-24T12:44:00.675 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:00 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2145097865' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:44:00.675 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:00 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:44:00.675 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:00 smithi090 ceph-mon[33646]: pgmap v1734: 188 pgs: 12 unknown, 176 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-03-24T12:44:01.188 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:44:01.192+0000 7f9455ffb640 1 -- 172.21.15.6:0/2145097865 <== mon.2 v2:172.21.15.6:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool f99c589a-562e-40f2-a1ef-11b16b6c29e6 v786) v1 ==== 223+0+0 (secure 0 0 0) 0x7f945005a420 con 0x7f94601053d0 2024-03-24T12:44:01.188 INFO:tasks.workunit.client.0.smithi006.stderr:set-quota max_objects = 10 for pool f99c589a-562e-40f2-a1ef-11b16b6c29e6 2024-03-24T12:44:01.191 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:44:01.195+0000 7f9466f56640 1 -- 172.21.15.6:0/2145097865 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f944806d5c0 msgr2=0x7f944806fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:44:01.191 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:44:01.195+0000 7f9466f56640 1 --2- 172.21.15.6:0/2145097865 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f944806d5c0 0x7f944806fa80 secure :-1 s=READY pgs=4120 cs=0 l=1 rev1=1 crypto rx=0x7f945c000c00 tx=0x7f945c0058d0 comp rx=0 tx=0).stop 2024-03-24T12:44:01.191 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:44:01.195+0000 7f9466f56640 1 -- 172.21.15.6:0/2145097865 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94601053d0 msgr2=0x7f9460197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:44:01.191 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:44:01.195+0000 7f9466f56640 1 --2- 172.21.15.6:0/2145097865 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94601053d0 0x7f9460197390 secure :-1 s=READY pgs=3092 cs=0 l=1 rev1=1 crypto rx=0x7f945000e9c0 tx=0x7f945000ee80 comp rx=0 tx=0).stop 2024-03-24T12:44:01.191 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:44:01.195+0000 7f9466f56640 1 -- 172.21.15.6:0/2145097865 shutdown_connections 2024-03-24T12:44:01.191 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:44:01.195+0000 7f9466f56640 1 --2- 172.21.15.6:0/2145097865 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f944806d5c0 0x7f944806fa80 unknown :-1 s=CLOSED pgs=4120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:44:01.191 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:44:01.195+0000 7f9466f56640 1 --2- 172.21.15.6:0/2145097865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f946019b710 0x7f946019bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:44:01.191 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:44:01.195+0000 7f9466f56640 1 --2- 172.21.15.6:0/2145097865 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94601053d0 0x7f9460197390 unknown :-1 s=CLOSED pgs=3092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:44:01.191 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:44:01.195+0000 7f9466f56640 1 --2- 172.21.15.6:0/2145097865 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9460100300 0x7f9460196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:44:01.191 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:44:01.195+0000 7f9466f56640 1 -- 172.21.15.6:0/2145097865 >> 172.21.15.6:0/2145097865 conn(0x7f94600771d0 msgr2=0x7f9460069800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:44:01.191 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:44:01.195+0000 7f9466f56640 1 -- 172.21.15.6:0/2145097865 shutdown_connections 2024-03-24T12:44:01.192 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:44:01.195+0000 7f9466f56640 1 -- 172.21.15.6:0/2145097865 wait complete. 2024-03-24T12:44:01.203 INFO:tasks.workunit.client.0.smithi006.stderr:+ sleep 30 2024-03-24T12:44:01.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "field": "max_objects", "val": "10"}]': finished 2024-03-24T12:44:01.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:01 smithi006 ceph-mon[30108]: osdmap e785: 8 total, 8 up, 8 in 2024-03-24T12:44:01.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:01 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2145097865' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:44:01.458 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:01 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:44:01.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "field": "max_objects", "val": "10"}]': finished 2024-03-24T12:44:01.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:01 smithi006 ceph-mon[41734]: osdmap e785: 8 total, 8 up, 8 in 2024-03-24T12:44:01.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:01 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2145097865' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:44:01.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:01 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:44:01.674 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "field": "max_objects", "val": "10"}]': finished 2024-03-24T12:44:01.674 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:01 smithi090 ceph-mon[33646]: osdmap e785: 8 total, 8 up, 8 in 2024-03-24T12:44:01.675 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:01 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2145097865' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:44:01.675 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:01 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "field": "max_objects", "val": "10"}]: dispatch 2024-03-24T12:44:02.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:02 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "field": "max_objects", "val": "10"}]': finished 2024-03-24T12:44:02.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:02 smithi006 ceph-mon[30108]: osdmap e786: 8 total, 8 up, 8 in 2024-03-24T12:44:02.457 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:02 smithi006 ceph-mon[30108]: pgmap v1737: 188 pgs: 12 unknown, 176 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:44:02.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:02 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "field": "max_objects", "val": "10"}]': finished 2024-03-24T12:44:02.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:02 smithi006 ceph-mon[41734]: osdmap e786: 8 total, 8 up, 8 in 2024-03-24T12:44:02.458 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:02 smithi006 ceph-mon[41734]: pgmap v1737: 188 pgs: 12 unknown, 176 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:44:02.674 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:02 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "field": "max_objects", "val": "10"}]': finished 2024-03-24T12:44:02.675 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:02 smithi090 ceph-mon[33646]: osdmap e786: 8 total, 8 up, 8 in 2024-03-24T12:44:02.675 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:02 smithi090 ceph-mon[33646]: pgmap v1737: 188 pgs: 12 unknown, 176 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail 2024-03-24T12:44:04.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:44:03 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:44:05.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:04 smithi090 ceph-mon[33646]: pgmap v1738: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 925 B/s rd, 0 op/s 2024-03-24T12:44:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:04 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:44:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:04 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:44:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:44:05.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:04 smithi006 ceph-mon[30108]: pgmap v1738: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 925 B/s rd, 0 op/s 2024-03-24T12:44:05.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:04 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:44:05.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:04 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:44:05.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:44:05.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:04 smithi006 ceph-mon[41734]: pgmap v1738: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 925 B/s rd, 0 op/s 2024-03-24T12:44:05.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:04 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:44:05.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:04 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:44:05.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:44:05.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:44:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:44:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:44:07.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:06 smithi090 ceph-mon[33646]: pgmap v1739: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 784 B/s rd, 0 op/s 2024-03-24T12:44:07.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:06 smithi090 ceph-mon[33646]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:44:07.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:06 smithi006 ceph-mon[30108]: pgmap v1739: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 784 B/s rd, 0 op/s 2024-03-24T12:44:07.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:06 smithi006 ceph-mon[30108]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:44:07.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:06 smithi006 ceph-mon[41734]: pgmap v1739: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 784 B/s rd, 0 op/s 2024-03-24T12:44:07.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:06 smithi006 ceph-mon[41734]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-24T12:44:09.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:08 smithi090 ceph-mon[33646]: pgmap v1740: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:44:09.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:08 smithi006 ceph-mon[30108]: pgmap v1740: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:44:09.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:08 smithi006 ceph-mon[41734]: pgmap v1740: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:44:11.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:10 smithi090 ceph-mon[33646]: pgmap v1741: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T12:44:11.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:10 smithi006 ceph-mon[30108]: pgmap v1741: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T12:44:11.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:10 smithi006 ceph-mon[41734]: pgmap v1741: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T12:44:13.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:12 smithi090 ceph-mon[33646]: pgmap v1742: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 972 B/s rd, 0 op/s 2024-03-24T12:44:13.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:12 smithi006 ceph-mon[30108]: pgmap v1742: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 972 B/s rd, 0 op/s 2024-03-24T12:44:13.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:12 smithi006 ceph-mon[41734]: pgmap v1742: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 972 B/s rd, 0 op/s 2024-03-24T12:44:14.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:44:13 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:44:15.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:14 smithi090 ceph-mon[33646]: pgmap v1743: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:44:15.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:14 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:44:15.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:14 smithi006 ceph-mon[30108]: pgmap v1743: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:44:15.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:14 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:44:15.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:14 smithi006 ceph-mon[41734]: pgmap v1743: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:44:15.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:14 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:44:15.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:44:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:44:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:44:17.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:16 smithi090 ceph-mon[33646]: pgmap v1744: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:17.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:16 smithi006 ceph-mon[30108]: pgmap v1744: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:17.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:16 smithi006 ceph-mon[41734]: pgmap v1744: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:19.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:18 smithi090 ceph-mon[33646]: pgmap v1745: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:44:19.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:18 smithi006 ceph-mon[41734]: pgmap v1745: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:44:19.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:18 smithi006 ceph-mon[30108]: pgmap v1745: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:44:20.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:44:20.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:44:20.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:44:21.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:20 smithi090 ceph-mon[33646]: pgmap v1746: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:44:21.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:20 smithi006 ceph-mon[30108]: pgmap v1746: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:44:21.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:20 smithi006 ceph-mon[41734]: pgmap v1746: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-24T12:44:23.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:22 smithi090 ceph-mon[33646]: pgmap v1747: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:23.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:22 smithi006 ceph-mon[30108]: pgmap v1747: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:23.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:22 smithi006 ceph-mon[41734]: pgmap v1747: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:24.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:44:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:44:25.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:24 smithi090 ceph-mon[33646]: pgmap v1748: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:44:25.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:24 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:44:25.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:24 smithi006 ceph-mon[30108]: pgmap v1748: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:44:25.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:24 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:44:25.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:24 smithi006 ceph-mon[41734]: pgmap v1748: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:44:25.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:24 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:44:25.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:44:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:44:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:44:27.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:26 smithi090 ceph-mon[33646]: pgmap v1749: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:27.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:26 smithi006 ceph-mon[30108]: pgmap v1749: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:27.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:26 smithi006 ceph-mon[41734]: pgmap v1749: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:29.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:28 smithi090 ceph-mon[33646]: pgmap v1750: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:44:29.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:28 smithi006 ceph-mon[30108]: pgmap v1750: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:44:29.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:28 smithi006 ceph-mon[41734]: pgmap v1750: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:44:31.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:30 smithi090 ceph-mon[33646]: pgmap v1751: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:31.204 INFO:tasks.workunit.client.0.smithi006.stderr:++ seq 1 10 2024-03-24T12:44:31.205 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in `seq 1 10` 2024-03-24T12:44:31.205 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p f99c589a-562e-40f2-a1ef-11b16b6c29e6 put obj1 /etc/passwd 2024-03-24T12:44:31.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:30 smithi006 ceph-mon[30108]: pgmap v1751: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:31.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:30 smithi006 ceph-mon[41734]: pgmap v1751: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:31.246 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in `seq 1 10` 2024-03-24T12:44:31.246 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p f99c589a-562e-40f2-a1ef-11b16b6c29e6 put obj2 /etc/passwd 2024-03-24T12:44:31.284 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in `seq 1 10` 2024-03-24T12:44:31.284 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p f99c589a-562e-40f2-a1ef-11b16b6c29e6 put obj3 /etc/passwd 2024-03-24T12:44:31.320 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in `seq 1 10` 2024-03-24T12:44:31.320 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p f99c589a-562e-40f2-a1ef-11b16b6c29e6 put obj4 /etc/passwd 2024-03-24T12:44:31.356 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in `seq 1 10` 2024-03-24T12:44:31.356 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p f99c589a-562e-40f2-a1ef-11b16b6c29e6 put obj5 /etc/passwd 2024-03-24T12:44:31.391 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in `seq 1 10` 2024-03-24T12:44:31.391 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p f99c589a-562e-40f2-a1ef-11b16b6c29e6 put obj6 /etc/passwd 2024-03-24T12:44:31.426 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in `seq 1 10` 2024-03-24T12:44:31.426 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p f99c589a-562e-40f2-a1ef-11b16b6c29e6 put obj7 /etc/passwd 2024-03-24T12:44:31.460 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in `seq 1 10` 2024-03-24T12:44:31.460 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p f99c589a-562e-40f2-a1ef-11b16b6c29e6 put obj8 /etc/passwd 2024-03-24T12:44:31.495 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in `seq 1 10` 2024-03-24T12:44:31.495 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p f99c589a-562e-40f2-a1ef-11b16b6c29e6 put obj9 /etc/passwd 2024-03-24T12:44:31.529 INFO:tasks.workunit.client.0.smithi006.stderr:+ for f in `seq 1 10` 2024-03-24T12:44:31.529 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p f99c589a-562e-40f2-a1ef-11b16b6c29e6 put obj10 /etc/passwd 2024-03-24T12:44:31.564 INFO:tasks.workunit.client.0.smithi006.stderr:+ sleep 30 2024-03-24T12:44:33.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:32 smithi090 ceph-mon[33646]: pgmap v1752: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:33.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:32 smithi006 ceph-mon[30108]: pgmap v1752: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:33.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:32 smithi006 ceph-mon[41734]: pgmap v1752: 188 pgs: 188 active+clean; 744 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:34.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:44:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:44:35.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:34 smithi090 ceph-mon[33646]: pgmap v1753: 188 pgs: 188 active+clean; 859 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-03-24T12:44:35.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:34 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:44:35.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:44:35.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:34 smithi006 ceph-mon[30108]: pgmap v1753: 188 pgs: 188 active+clean; 859 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-03-24T12:44:35.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:34 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:44:35.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:44:35.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:34 smithi006 ceph-mon[41734]: pgmap v1753: 188 pgs: 188 active+clean; 859 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-03-24T12:44:35.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:34 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:44:35.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:44:35.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:44:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:44:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:44:37.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:36 smithi090 ceph-mon[33646]: pgmap v1754: 188 pgs: 188 active+clean; 859 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-03-24T12:44:37.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:36 smithi090 ceph-mon[33646]: pool 'f99c589a-562e-40f2-a1ef-11b16b6c29e6' is full (reached quota's max_objects: 10) 2024-03-24T12:44:37.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:36 smithi090 ceph-mon[33646]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-24T12:44:37.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:36 smithi090 ceph-mon[33646]: osdmap e787: 8 total, 8 up, 8 in 2024-03-24T12:44:37.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:36 smithi006 ceph-mon[30108]: pgmap v1754: 188 pgs: 188 active+clean; 859 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-03-24T12:44:37.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:36 smithi006 ceph-mon[30108]: pool 'f99c589a-562e-40f2-a1ef-11b16b6c29e6' is full (reached quota's max_objects: 10) 2024-03-24T12:44:37.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:36 smithi006 ceph-mon[30108]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-24T12:44:37.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:36 smithi006 ceph-mon[30108]: osdmap e787: 8 total, 8 up, 8 in 2024-03-24T12:44:37.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:36 smithi006 ceph-mon[41734]: pgmap v1754: 188 pgs: 188 active+clean; 859 KiB data, 871 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-03-24T12:44:37.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:36 smithi006 ceph-mon[41734]: pool 'f99c589a-562e-40f2-a1ef-11b16b6c29e6' is full (reached quota's max_objects: 10) 2024-03-24T12:44:37.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:36 smithi006 ceph-mon[41734]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-24T12:44:37.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:36 smithi006 ceph-mon[41734]: osdmap e787: 8 total, 8 up, 8 in 2024-03-24T12:44:39.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:38 smithi090 ceph-mon[33646]: pgmap v1756: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-24T12:44:39.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:38 smithi006 ceph-mon[30108]: pgmap v1756: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-24T12:44:39.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:38 smithi006 ceph-mon[41734]: pgmap v1756: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-24T12:44:41.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:40 smithi090 ceph-mon[33646]: pgmap v1757: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-24T12:44:41.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:40 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:44:41.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:40 smithi006 ceph-mon[30108]: pgmap v1757: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-24T12:44:41.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:40 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:44:41.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:40 smithi006 ceph-mon[41734]: pgmap v1757: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-24T12:44:41.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:40 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-24T12:44:42.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:41 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:44:42.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:41 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:44:42.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:41 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:44:42.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:41 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:44:42.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:41 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:44:42.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:41 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:44:42.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:41 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-24T12:44:42.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:41 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-24T12:44:42.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:41 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:44:43.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:42 smithi090 ceph-mon[33646]: pgmap v1758: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-24T12:44:43.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:42 smithi006 ceph-mon[30108]: pgmap v1758: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-24T12:44:43.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:42 smithi006 ceph-mon[41734]: pgmap v1758: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-24T12:44:44.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:44:43 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:44:45.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:44 smithi090 ceph-mon[33646]: pgmap v1759: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:44:45.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:44 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:44:45.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:44 smithi006 ceph-mon[30108]: pgmap v1759: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:44:45.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:44 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:44:45.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:44 smithi006 ceph-mon[41734]: pgmap v1759: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:44:45.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:44 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:44:45.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:44:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:44:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:44:47.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:46 smithi090 ceph-mon[33646]: pgmap v1760: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:44:47.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:46 smithi006 ceph-mon[30108]: pgmap v1760: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:44:47.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:46 smithi006 ceph-mon[41734]: pgmap v1760: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:44:49.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:48 smithi090 ceph-mon[33646]: pgmap v1761: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T12:44:49.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:48 smithi006 ceph-mon[30108]: pgmap v1761: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T12:44:49.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:48 smithi006 ceph-mon[41734]: pgmap v1761: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T12:44:51.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:50 smithi090 ceph-mon[33646]: pgmap v1762: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:51.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:50 smithi090 ceph-mon[33646]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:44:51.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:50 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:44:51.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:50 smithi006 ceph-mon[30108]: pgmap v1762: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:51.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:50 smithi006 ceph-mon[30108]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:44:51.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:50 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:44:51.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:50 smithi006 ceph-mon[41734]: pgmap v1762: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:51.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:50 smithi006 ceph-mon[41734]: from='mgr.24464 ' entity='mgr.y' 2024-03-24T12:44:51.209 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:50 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:44:53.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:52 smithi090 ceph-mon[33646]: pgmap v1763: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:53.210 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:52 smithi006 ceph-mon[30108]: pgmap v1763: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:53.210 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:52 smithi006 ceph-mon[41734]: pgmap v1763: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:54.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:44:53 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:44:55.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:54 smithi090 ceph-mon[33646]: pgmap v1764: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:44:55.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:54 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:44:55.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:54 smithi006 ceph-mon[30108]: pgmap v1764: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:44:55.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:54 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:44:55.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:54 smithi006 ceph-mon[41734]: pgmap v1764: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:44:55.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:54 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:44:55.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:44:55 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:44:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:44:57.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:56 smithi090 ceph-mon[33646]: pgmap v1765: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:57.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:56 smithi006 ceph-mon[30108]: pgmap v1765: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:57.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:56 smithi006 ceph-mon[41734]: pgmap v1765: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:44:59.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:44:58 smithi090 ceph-mon[33646]: pgmap v1766: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:44:59.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:44:58 smithi006 ceph-mon[30108]: pgmap v1766: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:44:59.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:44:58 smithi006 ceph-mon[41734]: pgmap v1766: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:45:01.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:00 smithi090 ceph-mon[33646]: pgmap v1767: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:01.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:00 smithi006 ceph-mon[30108]: pgmap v1767: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:01.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:00 smithi006 ceph-mon[41734]: pgmap v1767: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:01.565 INFO:tasks.workunit.client.0.smithi006.stderr:+ rados -p 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 put threemore /etc/passwd 2024-03-24T12:45:01.606 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph osd pool set-quota 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 max_bytes 0 2024-03-24T12:45:01.698 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.700+0000 7f12c7f3e640 1 -- 172.21.15.6:0/699406327 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f12c0101b80 msgr2=0x7f12c010e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:45:01.698 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.700+0000 7f12c7f3e640 1 --2- 172.21.15.6:0/699406327 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f12c0101b80 0x7f12c010e080 secure :-1 s=READY pgs=3112 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009cd0 tx=0x7f12b802c0f0 comp rx=0 tx=0).stop 2024-03-24T12:45:01.698 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.700+0000 7f12c7f3e640 1 -- 172.21.15.6:0/699406327 shutdown_connections 2024-03-24T12:45:01.699 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.700+0000 7f12c7f3e640 1 --2- 172.21.15.6:0/699406327 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f12c0101b80 0x7f12c010e080 unknown :-1 s=CLOSED pgs=3112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:01.699 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.700+0000 7f12c7f3e640 1 --2- 172.21.15.6:0/699406327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c0101200 0x7f12c0101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:01.699 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.700+0000 7f12c7f3e640 1 --2- 172.21.15.6:0/699406327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12c0107250 0x7f12c0107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:01.699 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.700+0000 7f12c7f3e640 1 -- 172.21.15.6:0/699406327 >> 172.21.15.6:0/699406327 conn(0x7f12c00783a0 msgr2=0x7f12c00787b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:45:01.699 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.700+0000 7f12c7f3e640 1 -- 172.21.15.6:0/699406327 shutdown_connections 2024-03-24T12:45:01.699 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.700+0000 7f12c7f3e640 1 -- 172.21.15.6:0/699406327 wait complete. 2024-03-24T12:45:01.699 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.701+0000 7f12c7f3e640 1 Processor -- start 2024-03-24T12:45:01.699 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.701+0000 7f12c7f3e640 1 -- start start 2024-03-24T12:45:01.699 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.701+0000 7f12c7f3e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f12c0101200 0x7f12c019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:45:01.699 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.701+0000 7f12c7f3e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12c0101b80 0x7f12c019d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:45:01.699 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.701+0000 7f12c7f3e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c0107250 0x7f12c0196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:45:01.699 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.701+0000 7f12c7f3e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c01972b0 con 0x7f12c0101b80 2024-03-24T12:45:01.699 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.701+0000 7f12c7f3e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c0197420 con 0x7f12c0107250 2024-03-24T12:45:01.699 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.701+0000 7f12c7f3e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c01975e0 con 0x7f12c0101200 2024-03-24T12:45:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.701+0000 7f12c5cda640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f12c0101200 0x7f12c019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:45:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.701+0000 7f12c5cda640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f12c0101200 0x7f12c019cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:42456/0 (socket says 172.21.15.6:42456) 2024-03-24T12:45:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.701+0000 7f12c5cda640 1 -- 172.21.15.6:0/2870562626 learned_addr learned my addr 172.21.15.6:0/2870562626 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:45:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.701+0000 7f12c54d9640 1 --2- 172.21.15.6:0/2870562626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12c0101b80 0x7f12c019d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:45:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.701+0000 7f12c64db640 1 --2- 172.21.15.6:0/2870562626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c0107250 0x7f12c0196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:45:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.701+0000 7f12c5cda640 1 -- 172.21.15.6:0/2870562626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c0107250 msgr2=0x7f12c0196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:45:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.701+0000 7f12c5cda640 1 --2- 172.21.15.6:0/2870562626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c0107250 0x7f12c0196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.701+0000 7f12c5cda640 1 -- 172.21.15.6:0/2870562626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12c0101b80 msgr2=0x7f12c019d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:45:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.701+0000 7f12c5cda640 1 --2- 172.21.15.6:0/2870562626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12c0101b80 0x7f12c019d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.701+0000 7f12c5cda640 1 -- 172.21.15.6:0/2870562626 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12bc009490 con 0x7f12c0101200 2024-03-24T12:45:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.701+0000 7f12c64db640 1 --2- 172.21.15.6:0/2870562626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c0107250 0x7f12c0196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-24T12:45:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.701+0000 7f12c5cda640 1 --2- 172.21.15.6:0/2870562626 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f12c0101200 0x7f12c019cc70 secure :-1 s=READY pgs=3113 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00ea60 tx=0x7f12bc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:45:01.700 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.701+0000 7f12b6ffd640 1 -- 172.21.15.6:0/2870562626 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc00cde0 con 0x7f12c0101200 2024-03-24T12:45:01.701 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.702+0000 7f12b6ffd640 1 -- 172.21.15.6:0/2870562626 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f12bc0044a0 con 0x7f12c0101200 2024-03-24T12:45:01.701 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.702+0000 7f12c7f3e640 1 -- 172.21.15.6:0/2870562626 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b8009900 con 0x7f12c0101200 2024-03-24T12:45:01.701 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.702+0000 7f12b6ffd640 1 -- 172.21.15.6:0/2870562626 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc010430 con 0x7f12c0101200 2024-03-24T12:45:01.701 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.702+0000 7f12c7f3e640 1 -- 172.21.15.6:0/2870562626 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c01a3bf0 con 0x7f12c0101200 2024-03-24T12:45:01.701 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.703+0000 7f12c7f3e640 1 -- 172.21.15.6:0/2870562626 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c01028f0 con 0x7f12c0101200 2024-03-24T12:45:01.702 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.703+0000 7f12b6ffd640 1 -- 172.21.15.6:0/2870562626 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7f12bc003f60 con 0x7f12c0101200 2024-03-24T12:45:01.702 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.703+0000 7f12b6ffd640 1 --2- 172.21.15.6:0/2870562626 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f12ac06d5d0 0x7f12ac06fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:45:01.702 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.704+0000 7f12b6ffd640 1 -- 172.21.15.6:0/2870562626 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(787..787 src has 255..787) v4 ==== 9595+0+0 (secure 0 0 0) 0x7f12bc014070 con 0x7f12c0101200 2024-03-24T12:45:01.702 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.704+0000 7f12c54d9640 1 --2- 172.21.15.6:0/2870562626 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f12ac06d5d0 0x7f12ac06fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:45:01.702 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.704+0000 7f12b6ffd640 1 -- 172.21.15.6:0/2870562626 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=788}) v3 -- 0x7f12ac079b10 con 0x7f12c0101200 2024-03-24T12:45:01.702 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.704+0000 7f12c54d9640 1 --2- 172.21.15.6:0/2870562626 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f12ac06d5d0 0x7f12ac06fa90 secure :-1 s=READY pgs=4132 cs=0 l=1 rev1=1 crypto rx=0x7f12c0198510 tx=0x7f12b000b390 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:45:01.704 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.708+0000 7f12b6ffd640 1 -- 172.21.15.6:0/2870562626 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f12bc056de0 con 0x7f12c0101200 2024-03-24T12:45:01.891 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:01.895+0000 7f12c7f3e640 1 -- 172.21.15.6:0/2870562626 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f12c01981f0 con 0x7f12c0101200 2024-03-24T12:45:02.790 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:02.794+0000 7f12b6ffd640 1 -- 172.21.15.6:0/2870562626 <== mon.2 v2:172.21.15.6:3301/0 7 ==== osd_map(788..788 src has 255..788) v4 ==== 628+0+0 (secure 0 0 0) 0x7f12bc05b240 con 0x7f12c0101200 2024-03-24T12:45:02.790 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:02.794+0000 7f12b6ffd640 1 -- 172.21.15.6:0/2870562626 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=789}) v3 -- 0x7f12ac07a5b0 con 0x7f12c0101200 2024-03-24T12:45:02.795 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:02.800+0000 7f12b6ffd640 1 -- 172.21.15.6:0/2870562626 <== mon.2 v2:172.21.15.6:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 v788) v1 ==== 217+0+0 (secure 0 0 0) 0x7f12bc05a4a0 con 0x7f12c0101200 2024-03-24T12:45:02.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:02.914+0000 7f12c7f3e640 1 -- 172.21.15.6:0/2870562626 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f12c0078ec0 con 0x7f12c0101200 2024-03-24T12:45:03.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:02 smithi090 ceph-mon[33646]: pgmap v1768: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:03.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:02 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2870562626' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:45:03.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:02 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:45:03.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:02 smithi006 ceph-mon[41734]: pgmap v1768: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:03.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:02 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2870562626' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:45:03.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:02 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:45:03.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:02 smithi006 ceph-mon[30108]: pgmap v1768: 188 pgs: 188 active+clean; 859 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:03.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:02 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2870562626' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:45:03.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:02 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:45:03.796 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.800+0000 7f12b6ffd640 1 -- 172.21.15.6:0/2870562626 <== mon.2 v2:172.21.15.6:3301/0 9 ==== osd_map(789..789 src has 255..789) v4 ==== 628+0+0 (secure 0 0 0) 0x7f12bc05a4a0 con 0x7f12c0101200 2024-03-24T12:45:03.796 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.800+0000 7f12b6ffd640 1 -- 172.21.15.6:0/2870562626 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=790}) v3 -- 0x7f12ac07aa10 con 0x7f12c0101200 2024-03-24T12:45:03.802 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.807+0000 7f12b6ffd640 1 -- 172.21.15.6:0/2870562626 <== mon.2 v2:172.21.15.6:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 v789) v1 ==== 217+0+0 (secure 0 0 0) 0x7f12bc05a680 con 0x7f12c0101200 2024-03-24T12:45:03.803 INFO:tasks.workunit.client.0.smithi006.stderr:set-quota max_bytes = 0 for pool 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 2024-03-24T12:45:03.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.809+0000 7f12c7f3e640 1 -- 172.21.15.6:0/2870562626 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f12ac06d5d0 msgr2=0x7f12ac06fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:45:03.805 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.809+0000 7f12c7f3e640 1 --2- 172.21.15.6:0/2870562626 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f12ac06d5d0 0x7f12ac06fa90 secure :-1 s=READY pgs=4132 cs=0 l=1 rev1=1 crypto rx=0x7f12c0198510 tx=0x7f12b000b390 comp rx=0 tx=0).stop 2024-03-24T12:45:03.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.810+0000 7f12c7f3e640 1 -- 172.21.15.6:0/2870562626 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f12c0101200 msgr2=0x7f12c019cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:45:03.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.810+0000 7f12c7f3e640 1 --2- 172.21.15.6:0/2870562626 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f12c0101200 0x7f12c019cc70 secure :-1 s=READY pgs=3113 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00ea60 tx=0x7f12bc00ef20 comp rx=0 tx=0).stop 2024-03-24T12:45:03.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.810+0000 7f12c7f3e640 1 -- 172.21.15.6:0/2870562626 shutdown_connections 2024-03-24T12:45:03.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.810+0000 7f12c7f3e640 1 --2- 172.21.15.6:0/2870562626 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f12ac06d5d0 0x7f12ac06fa90 unknown :-1 s=CLOSED pgs=4132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:03.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.810+0000 7f12c7f3e640 1 --2- 172.21.15.6:0/2870562626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c0107250 0x7f12c0196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:03.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.810+0000 7f12c7f3e640 1 --2- 172.21.15.6:0/2870562626 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12c0101b80 0x7f12c019d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:03.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.810+0000 7f12c7f3e640 1 --2- 172.21.15.6:0/2870562626 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f12c0101200 0x7f12c019cc70 unknown :-1 s=CLOSED pgs=3113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:03.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.810+0000 7f12c7f3e640 1 -- 172.21.15.6:0/2870562626 >> 172.21.15.6:0/2870562626 conn(0x7f12c00783a0 msgr2=0x7f12c0104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:45:03.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.810+0000 7f12c7f3e640 1 -- 172.21.15.6:0/2870562626 shutdown_connections 2024-03-24T12:45:03.806 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.810+0000 7f12c7f3e640 1 -- 172.21.15.6:0/2870562626 wait complete. 2024-03-24T12:45:03.817 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph osd pool set-quota 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 max_objects 0 2024-03-24T12:45:03.908 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.910+0000 7effcfe96640 1 -- 172.21.15.6:0/2583635261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc8100300 msgr2=0x7effc81006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:45:03.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.910+0000 7effcfe96640 1 --2- 172.21.15.6:0/2583635261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc8100300 0x7effc81006e0 secure :-1 s=READY pgs=3121 cs=0 l=1 rev1=1 crypto rx=0x7effc4009930 tx=0x7effc402bc30 comp rx=0 tx=0).stop 2024-03-24T12:45:03.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.910+0000 7effcfe96640 1 -- 172.21.15.6:0/2583635261 shutdown_connections 2024-03-24T12:45:03.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.910+0000 7effcfe96640 1 --2- 172.21.15.6:0/2583635261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effc81053d0 0x7effc810cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:03.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.910+0000 7effcfe96640 1 --2- 172.21.15.6:0/2583635261 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7effc8100cb0 0x7effc8104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:03.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.910+0000 7effcfe96640 1 --2- 172.21.15.6:0/2583635261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc8100300 0x7effc81006e0 unknown :-1 s=CLOSED pgs=3121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:03.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.910+0000 7effcfe96640 1 -- 172.21.15.6:0/2583635261 >> 172.21.15.6:0/2583635261 conn(0x7effc80771d0 msgr2=0x7effc80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:45:03.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.910+0000 7effcfe96640 1 -- 172.21.15.6:0/2583635261 shutdown_connections 2024-03-24T12:45:03.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.910+0000 7effcfe96640 1 -- 172.21.15.6:0/2583635261 wait complete. 2024-03-24T12:45:03.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.911+0000 7effcfe96640 1 Processor -- start 2024-03-24T12:45:03.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.911+0000 7effcfe96640 1 -- start start 2024-03-24T12:45:03.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.911+0000 7effcfe96640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effc8100300 0x7effc8197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:45:03.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.911+0000 7effcfe96640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7effc81053d0 0x7effc8198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:45:03.909 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.911+0000 7effcfe96640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc81985f0 0x7effc81a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:45:03.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.911+0000 7effcfe96640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc81a3270 con 0x7effc81985f0 2024-03-24T12:45:03.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.911+0000 7effcfe96640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc81a33e0 con 0x7effc8100300 2024-03-24T12:45:03.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.911+0000 7effcfe96640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc81a35a0 con 0x7effc81053d0 2024-03-24T12:45:03.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.911+0000 7effcd431640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7effc81053d0 0x7effc8198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:45:03.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.911+0000 7effce433640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc81985f0 0x7effc81a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:45:03.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.911+0000 7effce433640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc81985f0 0x7effc81a2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:41556/0 (socket says 172.21.15.6:41556) 2024-03-24T12:45:03.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.911+0000 7effcd431640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7effc81053d0 0x7effc8198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:42472/0 (socket says 172.21.15.6:42472) 2024-03-24T12:45:03.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.911+0000 7effce433640 1 -- 172.21.15.6:0/3292342637 learned_addr learned my addr 172.21.15.6:0/3292342637 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:45:03.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.911+0000 7effcdc32640 1 --2- 172.21.15.6:0/3292342637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effc8100300 0x7effc8197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:45:03.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.912+0000 7effce433640 1 -- 172.21.15.6:0/3292342637 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7effc81053d0 msgr2=0x7effc8198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:45:03.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.912+0000 7effce433640 1 --2- 172.21.15.6:0/3292342637 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7effc81053d0 0x7effc8198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:03.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.912+0000 7effce433640 1 -- 172.21.15.6:0/3292342637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effc8100300 msgr2=0x7effc8197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:45:03.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.912+0000 7effce433640 1 --2- 172.21.15.6:0/3292342637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effc8100300 0x7effc8197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:03.910 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.912+0000 7effce433640 1 -- 172.21.15.6:0/3292342637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb8009490 con 0x7effc81985f0 2024-03-24T12:45:03.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.912+0000 7effcd431640 1 --2- 172.21.15.6:0/3292342637 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7effc81053d0 0x7effc8198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:45:03.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.912+0000 7effce433640 1 --2- 172.21.15.6:0/3292342637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc81985f0 0x7effc81a2ba0 secure :-1 s=READY pgs=3122 cs=0 l=1 rev1=1 crypto rx=0x7effc000cab0 tx=0x7effc000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:45:03.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.912+0000 7effbeffd640 1 -- 172.21.15.6:0/3292342637 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc000ae20 con 0x7effc81985f0 2024-03-24T12:45:03.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.912+0000 7effcfe96640 1 -- 172.21.15.6:0/3292342637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc4009560 con 0x7effc81985f0 2024-03-24T12:45:03.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.912+0000 7effbeffd640 1 -- 172.21.15.6:0/3292342637 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7effc000f8d0 con 0x7effc81985f0 2024-03-24T12:45:03.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.912+0000 7effbeffd640 1 -- 172.21.15.6:0/3292342637 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc0018830 con 0x7effc81985f0 2024-03-24T12:45:03.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.912+0000 7effcfe96640 1 -- 172.21.15.6:0/3292342637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc81a3bb0 con 0x7effc81985f0 2024-03-24T12:45:03.911 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.913+0000 7effcfe96640 1 -- 172.21.15.6:0/3292342637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc804ef50 con 0x7effc81985f0 2024-03-24T12:45:03.912 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.913+0000 7effbeffd640 1 -- 172.21.15.6:0/3292342637 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7effc000f450 con 0x7effc81985f0 2024-03-24T12:45:03.912 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.914+0000 7effbeffd640 1 --2- 172.21.15.6:0/3292342637 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7effb406d5c0 0x7effb406fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:45:03.912 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.914+0000 7effbeffd640 1 -- 172.21.15.6:0/3292342637 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(789..789 src has 255..789) v4 ==== 9595+0+0 (secure 0 0 0) 0x7effc008deb0 con 0x7effc81985f0 2024-03-24T12:45:03.912 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.914+0000 7effcdc32640 1 --2- 172.21.15.6:0/3292342637 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7effb406d5c0 0x7effb406fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:45:03.912 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.914+0000 7effbeffd640 1 -- 172.21.15.6:0/3292342637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=790}) v3 -- 0x7effb4079a80 con 0x7effc81985f0 2024-03-24T12:45:03.912 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.914+0000 7effcdc32640 1 --2- 172.21.15.6:0/3292342637 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7effb406d5c0 0x7effb406fa80 secure :-1 s=READY pgs=4133 cs=0 l=1 rev1=1 crypto rx=0x7effc402c6b0 tx=0x7effc4005fd0 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:45:03.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:03.919+0000 7effbeffd640 1 -- 172.21.15.6:0/3292342637 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7effc0056ce0 con 0x7effc81985f0 2024-03-24T12:45:04.097 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:04.102+0000 7effcfe96640 1 -- 172.21.15.6:0/3292342637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7effc8100cb0 con 0x7effc81985f0 2024-03-24T12:45:04.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:45:03 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:45:04.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:03 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]': finished 2024-03-24T12:45:04.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:03 smithi090 ceph-mon[33646]: osdmap e788: 8 total, 8 up, 8 in 2024-03-24T12:45:04.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:03 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/2870562626' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:45:04.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:03 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:45:04.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:03 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]': finished 2024-03-24T12:45:04.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:03 smithi006 ceph-mon[30108]: osdmap e788: 8 total, 8 up, 8 in 2024-03-24T12:45:04.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:03 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/2870562626' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:45:04.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:03 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:45:04.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:03 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]': finished 2024-03-24T12:45:04.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:03 smithi006 ceph-mon[41734]: osdmap e788: 8 total, 8 up, 8 in 2024-03-24T12:45:04.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:03 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/2870562626' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:45:04.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:03 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-24T12:45:04.797 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:04.800+0000 7effbeffd640 1 -- 172.21.15.6:0/3292342637 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 v790) v1 ==== 221+0+0 (secure 0 0 0) 0x7effc005a3a0 con 0x7effc81985f0 2024-03-24T12:45:04.797 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:04.801+0000 7effbeffd640 1 -- 172.21.15.6:0/3292342637 <== mon.0 v2:172.21.15.6:3300/0 8 ==== osd_map(790..790 src has 255..790) v4 ==== 628+0+0 (secure 0 0 0) 0x7effc008f020 con 0x7effc81985f0 2024-03-24T12:45:04.797 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:04.801+0000 7effbeffd640 1 -- 172.21.15.6:0/3292342637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=791}) v3 -- 0x7effb407a9e0 con 0x7effc81985f0 2024-03-24T12:45:04.914 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:04.918+0000 7effcfe96640 1 -- 172.21.15.6:0/3292342637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7effc8100e40 con 0x7effc81985f0 2024-03-24T12:45:05.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:04 smithi090 ceph-mon[33646]: pgmap v1770: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-03-24T12:45:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:04 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]': finished 2024-03-24T12:45:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:04 smithi090 ceph-mon[33646]: osdmap e789: 8 total, 8 up, 8 in 2024-03-24T12:45:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:04 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3292342637' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-24T12:45:05.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:04 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:45:05.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:04 smithi006 ceph-mon[30108]: pgmap v1770: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-03-24T12:45:05.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:04 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]': finished 2024-03-24T12:45:05.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:04 smithi006 ceph-mon[30108]: osdmap e789: 8 total, 8 up, 8 in 2024-03-24T12:45:05.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:04 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3292342637' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-24T12:45:05.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:04 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:45:05.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:04 smithi006 ceph-mon[41734]: pgmap v1770: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-03-24T12:45:05.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:04 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_bytes", "val": "0"}]': finished 2024-03-24T12:45:05.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:04 smithi006 ceph-mon[41734]: osdmap e789: 8 total, 8 up, 8 in 2024-03-24T12:45:05.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:04 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3292342637' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-24T12:45:05.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:04 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:45:05.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:45:05 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:45:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:45:05.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:05.815+0000 7effbeffd640 1 -- 172.21.15.6:0/3292342637 <== mon.0 v2:172.21.15.6:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 v791) v1 ==== 221+0+0 (secure 0 0 0) 0x7effc005a580 con 0x7effc81985f0 2024-03-24T12:45:05.811 INFO:tasks.workunit.client.0.smithi006.stderr:set-quota max_objects = 0 for pool 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 2024-03-24T12:45:05.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:05.815+0000 7effbeffd640 1 -- 172.21.15.6:0/3292342637 <== mon.0 v2:172.21.15.6:3300/0 10 ==== osd_map(791..791 src has 255..791) v4 ==== 628+0+0 (secure 0 0 0) 0x7effc000f700 con 0x7effc81985f0 2024-03-24T12:45:05.811 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:05.815+0000 7effbeffd640 1 -- 172.21.15.6:0/3292342637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=792}) v3 -- 0x7effb407aca0 con 0x7effc81985f0 2024-03-24T12:45:05.815 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:05.819+0000 7effcfe96640 1 -- 172.21.15.6:0/3292342637 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7effb406d5c0 msgr2=0x7effb406fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:45:05.815 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:05.819+0000 7effcfe96640 1 --2- 172.21.15.6:0/3292342637 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7effb406d5c0 0x7effb406fa80 secure :-1 s=READY pgs=4133 cs=0 l=1 rev1=1 crypto rx=0x7effc402c6b0 tx=0x7effc4005fd0 comp rx=0 tx=0).stop 2024-03-24T12:45:05.815 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:05.819+0000 7effcfe96640 1 -- 172.21.15.6:0/3292342637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc81985f0 msgr2=0x7effc81a2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:45:05.815 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:05.819+0000 7effcfe96640 1 --2- 172.21.15.6:0/3292342637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc81985f0 0x7effc81a2ba0 secure :-1 s=READY pgs=3122 cs=0 l=1 rev1=1 crypto rx=0x7effc000cab0 tx=0x7effc000cf70 comp rx=0 tx=0).stop 2024-03-24T12:45:05.815 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:05.819+0000 7effcfe96640 1 -- 172.21.15.6:0/3292342637 shutdown_connections 2024-03-24T12:45:05.816 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:05.819+0000 7effcfe96640 1 --2- 172.21.15.6:0/3292342637 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7effb406d5c0 0x7effb406fa80 unknown :-1 s=CLOSED pgs=4133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:05.816 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:05.819+0000 7effcfe96640 1 --2- 172.21.15.6:0/3292342637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effc81985f0 0x7effc81a2ba0 unknown :-1 s=CLOSED pgs=3122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:05.816 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:05.819+0000 7effcfe96640 1 --2- 172.21.15.6:0/3292342637 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7effc81053d0 0x7effc8198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:05.816 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:05.819+0000 7effcfe96640 1 --2- 172.21.15.6:0/3292342637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effc8100300 0x7effc8197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:05.816 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:05.819+0000 7effcfe96640 1 -- 172.21.15.6:0/3292342637 >> 172.21.15.6:0/3292342637 conn(0x7effc80771d0 msgr2=0x7effc80fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:45:05.816 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:05.820+0000 7effcfe96640 1 -- 172.21.15.6:0/3292342637 shutdown_connections 2024-03-24T12:45:05.816 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:05.820+0000 7effcfe96640 1 -- 172.21.15.6:0/3292342637 wait complete. 2024-03-24T12:45:05.829 INFO:tasks.workunit.client.0.smithi006.stderr:+ sleep 30 2024-03-24T12:45:06.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:05 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:45:06.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:05 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3292342637' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]': finished 2024-03-24T12:45:06.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:05 smithi090 ceph-mon[33646]: osdmap e790: 8 total, 8 up, 8 in 2024-03-24T12:45:06.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:05 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3292342637' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-24T12:45:06.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:05 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:45:06.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:05 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3292342637' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]': finished 2024-03-24T12:45:06.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:05 smithi006 ceph-mon[30108]: osdmap e790: 8 total, 8 up, 8 in 2024-03-24T12:45:06.208 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:05 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3292342637' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-24T12:45:06.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:05 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:45:06.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:05 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3292342637' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]': finished 2024-03-24T12:45:06.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:05 smithi006 ceph-mon[41734]: osdmap e790: 8 total, 8 up, 8 in 2024-03-24T12:45:06.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:05 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3292342637' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-24T12:45:07.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:06 smithi090 ceph-mon[33646]: pgmap v1773: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-03-24T12:45:07.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:06 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3292342637' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]': finished 2024-03-24T12:45:07.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:06 smithi090 ceph-mon[33646]: osdmap e791: 8 total, 8 up, 8 in 2024-03-24T12:45:07.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:06 smithi006 ceph-mon[30108]: pgmap v1773: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-03-24T12:45:07.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:06 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3292342637' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]': finished 2024-03-24T12:45:07.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:06 smithi006 ceph-mon[30108]: osdmap e791: 8 total, 8 up, 8 in 2024-03-24T12:45:07.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:06 smithi006 ceph-mon[41734]: pgmap v1773: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-03-24T12:45:07.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:06 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3292342637' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "field": "max_objects", "val": "0"}]': finished 2024-03-24T12:45:07.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:06 smithi006 ceph-mon[41734]: osdmap e791: 8 total, 8 up, 8 in 2024-03-24T12:45:09.177 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:08 smithi090 ceph-mon[33646]: pgmap v1775: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T12:45:09.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:08 smithi006 ceph-mon[30108]: pgmap v1775: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T12:45:09.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:08 smithi006 ceph-mon[41734]: pgmap v1775: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-24T12:45:11.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:10 smithi090 ceph-mon[33646]: pgmap v1776: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:11.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:10 smithi006 ceph-mon[30108]: pgmap v1776: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:11.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:10 smithi006 ceph-mon[41734]: pgmap v1776: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:13.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:12 smithi090 ceph-mon[33646]: pgmap v1777: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 645 B/s rd, 0 op/s 2024-03-24T12:45:13.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:12 smithi006 ceph-mon[30108]: pgmap v1777: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 645 B/s rd, 0 op/s 2024-03-24T12:45:13.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:12 smithi006 ceph-mon[41734]: pgmap v1777: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 645 B/s rd, 0 op/s 2024-03-24T12:45:14.425 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:45:13 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:45:15.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:14 smithi090 ceph-mon[33646]: pgmap v1778: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T12:45:15.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:14 smithi006 ceph-mon[30108]: pgmap v1778: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T12:45:15.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:14 smithi006 ceph-mon[41734]: pgmap v1778: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-24T12:45:15.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:45:15 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:45:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:45:16.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:15 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:45:16.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:15 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:45:16.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:15 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:45:17.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:16 smithi090 ceph-mon[33646]: pgmap v1779: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:45:17.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:16 smithi006 ceph-mon[30108]: pgmap v1779: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:45:17.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:16 smithi006 ceph-mon[41734]: pgmap v1779: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:45:19.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:18 smithi090 ceph-mon[33646]: pgmap v1780: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T12:45:19.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:18 smithi006 ceph-mon[30108]: pgmap v1780: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T12:45:19.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:18 smithi006 ceph-mon[41734]: pgmap v1780: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-24T12:45:20.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:19 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:45:20.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:19 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:45:20.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:19 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:45:21.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:20 smithi090 ceph-mon[33646]: pgmap v1781: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:21.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:20 smithi006 ceph-mon[30108]: pgmap v1781: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:21.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:20 smithi006 ceph-mon[41734]: pgmap v1781: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:23.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:22 smithi090 ceph-mon[33646]: pgmap v1782: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:23.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:22 smithi006 ceph-mon[30108]: pgmap v1782: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:23.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:22 smithi006 ceph-mon[41734]: pgmap v1782: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:24.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:45:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:45:25.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:24 smithi090 ceph-mon[33646]: pgmap v1783: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:45:25.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:24 smithi006 ceph-mon[30108]: pgmap v1783: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:45:25.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:24 smithi006 ceph-mon[41734]: pgmap v1783: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:45:25.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:45:25 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:45:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:45:26.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:25 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:45:26.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:25 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:45:26.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:25 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:45:27.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:26 smithi090 ceph-mon[33646]: pgmap v1784: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:27.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:26 smithi006 ceph-mon[30108]: pgmap v1784: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:27.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:26 smithi006 ceph-mon[41734]: pgmap v1784: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:29.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:28 smithi090 ceph-mon[33646]: pgmap v1785: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:45:29.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:28 smithi006 ceph-mon[30108]: pgmap v1785: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:45:29.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:28 smithi006 ceph-mon[41734]: pgmap v1785: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:45:31.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:30 smithi090 ceph-mon[33646]: pgmap v1786: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:31.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:30 smithi006 ceph-mon[30108]: pgmap v1786: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:31.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:30 smithi006 ceph-mon[41734]: pgmap v1786: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:33.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:32 smithi090 ceph-mon[33646]: pgmap v1787: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:33.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:32 smithi006 ceph-mon[30108]: pgmap v1787: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:33.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:32 smithi006 ceph-mon[41734]: pgmap v1787: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:34.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:45:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug there is no tcmu-runner data available 2024-03-24T12:45:35.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:34 smithi090 ceph-mon[33646]: pgmap v1788: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:45:35.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:34 smithi090 ceph-mon[33646]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:45:35.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:34 smithi006 ceph-mon[30108]: pgmap v1788: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:45:35.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:34 smithi006 ceph-mon[30108]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:45:35.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:34 smithi006 ceph-mon[41734]: pgmap v1788: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-24T12:45:35.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:34 smithi006 ceph-mon[41734]: from='mgr.24464 172.21.15.6:0/90669099' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-24T12:45:35.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:45:35 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: ::ffff:172.21.15.90 - - [24/Mar/2024:12:45:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-24T12:45:35.830 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph osd pool delete 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 54e008b8-0c65-4b2f-9ae3-d0f6346347d3 --yes-i-really-really-mean-it 2024-03-24T12:45:35.926 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.929+0000 7f79d97d0640 1 -- 172.21.15.6:0/170153900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f79d4102d30 msgr2=0x7f79d410b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:45:35.926 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.929+0000 7f79d97d0640 1 --2- 172.21.15.6:0/170153900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f79d4102d30 0x7f79d410b210 secure :-1 s=READY pgs=3114 cs=0 l=1 rev1=1 crypto rx=0x7f79c8009860 tx=0x7f79c802bc00 comp rx=0 tx=0).stop 2024-03-24T12:45:35.926 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.930+0000 7f79d97d0640 1 -- 172.21.15.6:0/170153900 shutdown_connections 2024-03-24T12:45:35.926 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.930+0000 7f79d97d0640 1 --2- 172.21.15.6:0/170153900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79d410b7e0 0x7f79d410dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:35.926 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.930+0000 7f79d97d0640 1 --2- 172.21.15.6:0/170153900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f79d4102d30 0x7f79d410b210 unknown :-1 s=CLOSED pgs=3114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:35.926 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.930+0000 7f79d97d0640 1 --2- 172.21.15.6:0/170153900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79d4102410 0x7f79d41027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:35.926 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.930+0000 7f79d97d0640 1 -- 172.21.15.6:0/170153900 >> 172.21.15.6:0/170153900 conn(0x7f79d4076400 msgr2=0x7f79d4076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:45:35.926 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.930+0000 7f79d97d0640 1 -- 172.21.15.6:0/170153900 shutdown_connections 2024-03-24T12:45:35.926 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.930+0000 7f79d97d0640 1 -- 172.21.15.6:0/170153900 wait complete. 2024-03-24T12:45:35.926 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.930+0000 7f79d97d0640 1 Processor -- start 2024-03-24T12:45:35.926 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.930+0000 7f79d97d0640 1 -- start start 2024-03-24T12:45:35.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.931+0000 7f79d97d0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f79d4102410 0x7f79d4196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:45:35.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.931+0000 7f79d97d0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79d4102d30 0x7f79d41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:45:35.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.931+0000 7f79d97d0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79d410b7e0 0x7f79d419b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:45:35.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.931+0000 7f79d97d0640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d419be60 con 0x7f79d410b7e0 2024-03-24T12:45:35.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.931+0000 7f79d97d0640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d419bfd0 con 0x7f79d4102d30 2024-03-24T12:45:35.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.931+0000 7f79d97d0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d419c190 con 0x7f79d4102410 2024-03-24T12:45:35.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.931+0000 7f79d2ffd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f79d4102410 0x7f79d4196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:45:35.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.931+0000 7f79d37fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79d410b7e0 0x7f79d419b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:45:35.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.931+0000 7f79d2ffd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f79d4102410 0x7f79d4196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:43864/0 (socket says 172.21.15.6:43864) 2024-03-24T12:45:35.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.931+0000 7f79d2ffd640 1 -- 172.21.15.6:0/3504250001 learned_addr learned my addr 172.21.15.6:0/3504250001 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:45:35.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.931+0000 7f79d37fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79d410b7e0 0x7f79d419b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56774/0 (socket says 172.21.15.6:56774) 2024-03-24T12:45:35.929 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.931+0000 7f79d27fc640 1 --2- 172.21.15.6:0/3504250001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79d4102d30 0x7f79d41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:45:35.930 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.931+0000 7f79d37fe640 1 -- 172.21.15.6:0/3504250001 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f79d4102410 msgr2=0x7f79d4196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:45:35.930 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.931+0000 7f79d37fe640 1 --2- 172.21.15.6:0/3504250001 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f79d4102410 0x7f79d4196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:35.930 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.931+0000 7f79d37fe640 1 -- 172.21.15.6:0/3504250001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79d4102d30 msgr2=0x7f79d41972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:45:35.930 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.931+0000 7f79d37fe640 1 --2- 172.21.15.6:0/3504250001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79d4102d30 0x7f79d41972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:35.930 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.931+0000 7f79d37fe640 1 -- 172.21.15.6:0/3504250001 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79cc009900 con 0x7f79d410b7e0 2024-03-24T12:45:35.930 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.931+0000 7f79d2ffd640 1 --2- 172.21.15.6:0/3504250001 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f79d4102410 0x7f79d4196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-24T12:45:35.930 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.931+0000 7f79d37fe640 1 --2- 172.21.15.6:0/3504250001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79d410b7e0 0x7f79d419b860 secure :-1 s=READY pgs=3123 cs=0 l=1 rev1=1 crypto rx=0x7f79cc005fd0 tx=0x7f79cc00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:45:35.930 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.931+0000 7f79bbfff640 1 -- 172.21.15.6:0/3504250001 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc003970 con 0x7f79d410b7e0 2024-03-24T12:45:35.930 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.932+0000 7f79d97d0640 1 -- 172.21.15.6:0/3504250001 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c8009490 con 0x7f79d410b7e0 2024-03-24T12:45:35.930 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.932+0000 7f79bbfff640 1 -- 172.21.15.6:0/3504250001 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f79cc004020 con 0x7f79d410b7e0 2024-03-24T12:45:35.930 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.932+0000 7f79bbfff640 1 -- 172.21.15.6:0/3504250001 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc00b870 con 0x7f79d410b7e0 2024-03-24T12:45:35.930 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.932+0000 7f79d97d0640 1 -- 172.21.15.6:0/3504250001 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d41a3b10 con 0x7f79d410b7e0 2024-03-24T12:45:35.930 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.933+0000 7f79d97d0640 1 -- 172.21.15.6:0/3504250001 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d4076f20 con 0x7f79d410b7e0 2024-03-24T12:45:35.930 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.933+0000 7f79bbfff640 1 -- 172.21.15.6:0/3504250001 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7f79cc00a040 con 0x7f79d410b7e0 2024-03-24T12:45:35.931 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.934+0000 7f79bbfff640 1 --2- 172.21.15.6:0/3504250001 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f79bc06d5d0 0x7f79bc06fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:45:35.931 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.934+0000 7f79d2ffd640 1 --2- 172.21.15.6:0/3504250001 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f79bc06d5d0 0x7f79bc06fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:45:35.931 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.934+0000 7f79bbfff640 1 -- 172.21.15.6:0/3504250001 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(791..791 src has 255..791) v4 ==== 9595+0+0 (secure 0 0 0) 0x7f79cc08dc60 con 0x7f79d410b7e0 2024-03-24T12:45:35.932 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.934+0000 7f79bbfff640 1 -- 172.21.15.6:0/3504250001 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=792}) v3 -- 0x7f79bc079a90 con 0x7f79d410b7e0 2024-03-24T12:45:35.932 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.934+0000 7f79d2ffd640 1 --2- 172.21.15.6:0/3504250001 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f79bc06d5d0 0x7f79bc06fa90 secure :-1 s=READY pgs=4134 cs=0 l=1 rev1=1 crypto rx=0x7f79c40058c0 tx=0x7f79c400a380 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:45:35.934 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:35.939+0000 7f79bbfff640 1 -- 172.21.15.6:0/3504250001 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f79cc056af0 con 0x7f79d410b7e0 2024-03-24T12:45:36.119 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:36.124+0000 7f79d97d0640 1 -- 172.21.15.6:0/3504250001 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pool2": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f79d404efc0 con 0x7f79d410b7e0 2024-03-24T12:45:36.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:35 smithi090 ceph-mon[33646]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:45:36.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:35 smithi006 ceph-mon[30108]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:45:36.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:35 smithi006 ceph-mon[41734]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-24T12:45:36.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:36.883+0000 7f79bbfff640 1 -- 172.21.15.6:0/3504250001 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pool2": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "yes_i_really_really_mean_it": true}]=0 pool '54e008b8-0c65-4b2f-9ae3-d0f6346347d3' removed v792) v1 ==== 248+0+0 (secure 0 0 0) 0x7f79cc05a1b0 con 0x7f79d410b7e0 2024-03-24T12:45:36.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:36.884+0000 7f79bbfff640 1 -- 172.21.15.6:0/3504250001 <== mon.0 v2:172.21.15.6:3300/0 8 ==== osd_map(792..792 src has 255..792) v4 ==== 296+0+0 (secure 0 0 0) 0x7f79cc05e8f0 con 0x7f79d410b7e0 2024-03-24T12:45:36.880 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:36.884+0000 7f79bbfff640 1 -- 172.21.15.6:0/3504250001 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=793}) v3 -- 0x7f79bc07a710 con 0x7f79d410b7e0 2024-03-24T12:45:37.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.006+0000 7f79d97d0640 1 -- 172.21.15.6:0/3504250001 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pool2": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f79d4105fe0 con 0x7f79d410b7e0 2024-03-24T12:45:37.004 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.007+0000 7f79bbfff640 1 -- 172.21.15.6:0/3504250001 <== mon.0 v2:172.21.15.6:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pool2": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "yes_i_really_really_mean_it": true}]=0 pool '54e008b8-0c65-4b2f-9ae3-d0f6346347d3' does not exist v792) v1 ==== 255+0+0 (secure 0 0 0) 0x7f79cc05a390 con 0x7f79d410b7e0 2024-03-24T12:45:37.004 INFO:tasks.workunit.client.0.smithi006.stderr:pool '54e008b8-0c65-4b2f-9ae3-d0f6346347d3' does not exist 2024-03-24T12:45:37.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.009+0000 7f79d97d0640 1 -- 172.21.15.6:0/3504250001 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f79bc06d5d0 msgr2=0x7f79bc06fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:45:37.006 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.009+0000 7f79d97d0640 1 --2- 172.21.15.6:0/3504250001 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f79bc06d5d0 0x7f79bc06fa90 secure :-1 s=READY pgs=4134 cs=0 l=1 rev1=1 crypto rx=0x7f79c40058c0 tx=0x7f79c400a380 comp rx=0 tx=0).stop 2024-03-24T12:45:37.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.009+0000 7f79d97d0640 1 -- 172.21.15.6:0/3504250001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79d410b7e0 msgr2=0x7f79d419b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:45:37.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.009+0000 7f79d97d0640 1 --2- 172.21.15.6:0/3504250001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79d410b7e0 0x7f79d419b860 secure :-1 s=READY pgs=3123 cs=0 l=1 rev1=1 crypto rx=0x7f79cc005fd0 tx=0x7f79cc00c800 comp rx=0 tx=0).stop 2024-03-24T12:45:37.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.009+0000 7f79d97d0640 1 -- 172.21.15.6:0/3504250001 shutdown_connections 2024-03-24T12:45:37.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.009+0000 7f79d97d0640 1 --2- 172.21.15.6:0/3504250001 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7f79bc06d5d0 0x7f79bc06fa90 unknown :-1 s=CLOSED pgs=4134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:37.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.009+0000 7f79d97d0640 1 --2- 172.21.15.6:0/3504250001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79d410b7e0 0x7f79d419b860 unknown :-1 s=CLOSED pgs=3123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:37.007 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.009+0000 7f79d97d0640 1 --2- 172.21.15.6:0/3504250001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79d4102d30 0x7f79d41972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:37.008 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.009+0000 7f79d97d0640 1 --2- 172.21.15.6:0/3504250001 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f79d4102410 0x7f79d4196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:37.008 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.009+0000 7f79d97d0640 1 -- 172.21.15.6:0/3504250001 >> 172.21.15.6:0/3504250001 conn(0x7f79d4076400 msgr2=0x7f79d41031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:45:37.008 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.009+0000 7f79d97d0640 1 -- 172.21.15.6:0/3504250001 shutdown_connections 2024-03-24T12:45:37.008 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.009+0000 7f79d97d0640 1 -- 172.21.15.6:0/3504250001 wait complete. 2024-03-24T12:45:37.015 INFO:tasks.workunit.client.0.smithi006.stderr:+ ceph osd pool delete f99c589a-562e-40f2-a1ef-11b16b6c29e6 f99c589a-562e-40f2-a1ef-11b16b6c29e6 --yes-i-really-really-mean-it 2024-03-24T12:45:37.104 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.105+0000 7fd541de2640 1 -- 172.21.15.6:0/1771181135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd53c102d30 msgr2=0x7fd53c10b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:45:37.104 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.105+0000 7fd541de2640 1 --2- 172.21.15.6:0/1771181135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd53c102d30 0x7fd53c10b210 secure :-1 s=READY pgs=3124 cs=0 l=1 rev1=1 crypto rx=0x7fd524009900 tx=0x7fd52402bcd0 comp rx=0 tx=0).stop 2024-03-24T12:45:37.104 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.105+0000 7fd541de2640 1 -- 172.21.15.6:0/1771181135 shutdown_connections 2024-03-24T12:45:37.105 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.105+0000 7fd541de2640 1 --2- 172.21.15.6:0/1771181135 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd53c10b7e0 0x7fd53c10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:37.105 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.105+0000 7fd541de2640 1 --2- 172.21.15.6:0/1771181135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd53c102d30 0x7fd53c10b210 unknown :-1 s=CLOSED pgs=3124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:37.105 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.105+0000 7fd541de2640 1 --2- 172.21.15.6:0/1771181135 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd53c102410 0x7fd53c1027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:37.105 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.105+0000 7fd541de2640 1 -- 172.21.15.6:0/1771181135 >> 172.21.15.6:0/1771181135 conn(0x7fd53c076400 msgr2=0x7fd53c076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:45:37.105 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.105+0000 7fd541de2640 1 -- 172.21.15.6:0/1771181135 shutdown_connections 2024-03-24T12:45:37.105 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.105+0000 7fd541de2640 1 -- 172.21.15.6:0/1771181135 wait complete. 2024-03-24T12:45:37.105 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.105+0000 7fd541de2640 1 Processor -- start 2024-03-24T12:45:37.105 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.105+0000 7fd541de2640 1 -- start start 2024-03-24T12:45:37.105 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.106+0000 7fd541de2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd53c102410 0x7fd53c196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:45:37.105 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.106+0000 7fd541de2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd53c102d30 0x7fd53c197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:45:37.105 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.106+0000 7fd541de2640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd53c10b7e0 0x7fd53c19b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:45:37.105 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.106+0000 7fd541de2640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c19be20 con 0x7fd53c10b7e0 2024-03-24T12:45:37.105 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.106+0000 7fd541de2640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c19bf90 con 0x7fd53c102410 2024-03-24T12:45:37.105 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.106+0000 7fd541de2640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c19c150 con 0x7fd53c102d30 2024-03-24T12:45:37.106 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.106+0000 7fd532ffd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd53c102d30 0x7fd53c197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:45:37.106 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.106+0000 7fd53bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd53c10b7e0 0x7fd53c19b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:45:37.106 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.106+0000 7fd532ffd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd53c102d30 0x7fd53c197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:43876/0 (socket says 172.21.15.6:43876) 2024-03-24T12:45:37.106 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.106+0000 7fd532ffd640 1 -- 172.21.15.6:0/809583616 learned_addr learned my addr 172.21.15.6:0/809583616 (peer_addr_for_me v2:172.21.15.6:0/0) 2024-03-24T12:45:37.106 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.106+0000 7fd53bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd53c10b7e0 0x7fd53c19b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56792/0 (socket says 172.21.15.6:56792) 2024-03-24T12:45:37.106 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.106+0000 7fd53b7fe640 1 --2- 172.21.15.6:0/809583616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd53c102410 0x7fd53c196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:45:37.106 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.106+0000 7fd532ffd640 1 -- 172.21.15.6:0/809583616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd53c102410 msgr2=0x7fd53c196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:45:37.106 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.106+0000 7fd532ffd640 1 --2- 172.21.15.6:0/809583616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd53c102410 0x7fd53c196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:37.106 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.106+0000 7fd532ffd640 1 -- 172.21.15.6:0/809583616 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd53c10b7e0 msgr2=0x7fd53c19b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:45:37.106 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.106+0000 7fd532ffd640 1 --2- 172.21.15.6:0/809583616 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd53c10b7e0 0x7fd53c19b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:37.106 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.106+0000 7fd532ffd640 1 -- 172.21.15.6:0/809583616 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd534009840 con 0x7fd53c102d30 2024-03-24T12:45:37.107 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.106+0000 7fd532ffd640 1 --2- 172.21.15.6:0/809583616 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd53c102d30 0x7fd53c197290 secure :-1 s=READY pgs=3115 cs=0 l=1 rev1=1 crypto rx=0x7fd524006010 tx=0x7fd5240042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:45:37.107 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.106+0000 7fd5397fa640 1 -- 172.21.15.6:0/809583616 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd524004aa0 con 0x7fd53c102d30 2024-03-24T12:45:37.107 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.106+0000 7fd5397fa640 1 -- 172.21.15.6:0/809583616 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd5240044e0 con 0x7fd53c102d30 2024-03-24T12:45:37.107 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.106+0000 7fd541de2640 1 -- 172.21.15.6:0/809583616 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd524009560 con 0x7fd53c102d30 2024-03-24T12:45:37.107 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.106+0000 7fd5397fa640 1 -- 172.21.15.6:0/809583616 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd524005360 con 0x7fd53c102d30 2024-03-24T12:45:37.107 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.106+0000 7fd541de2640 1 -- 172.21.15.6:0/809583616 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd53c1a3ad0 con 0x7fd53c102d30 2024-03-24T12:45:37.108 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.107+0000 7fd541de2640 1 -- 172.21.15.6:0/809583616 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd53c076f20 con 0x7fd53c102d30 2024-03-24T12:45:37.108 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.108+0000 7fd5397fa640 1 -- 172.21.15.6:0/809583616 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 21) v1 ==== 91135+0+0 (secure 0 0 0) 0x7fd5240054c0 con 0x7fd53c102d30 2024-03-24T12:45:37.108 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.108+0000 7fd5397fa640 1 --2- 172.21.15.6:0/809583616 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fd52806d5d0 0x7fd52806fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-24T12:45:37.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.108+0000 7fd53b7fe640 1 --2- 172.21.15.6:0/809583616 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fd52806d5d0 0x7fd52806fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-24T12:45:37.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.108+0000 7fd5397fa640 1 -- 172.21.15.6:0/809583616 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(792..792 src has 255..792) v4 ==== 9207+0+0 (secure 0 0 0) 0x7fd5240b03d0 con 0x7fd53c102d30 2024-03-24T12:45:37.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.109+0000 7fd5397fa640 1 -- 172.21.15.6:0/809583616 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=793}) v3 -- 0x7fd528079270 con 0x7fd53c102d30 2024-03-24T12:45:37.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.109+0000 7fd53b7fe640 1 --2- 172.21.15.6:0/809583616 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fd52806d5d0 0x7fd52806fa90 secure :-1 s=READY pgs=4135 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009bd0 tx=0x7fd52c005f90 comp rx=0 tx=0).ready entity=mgr.24464 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-24T12:45:37.109 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.113+0000 7fd5397fa640 1 -- 172.21.15.6:0/809583616 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd524079450 con 0x7fd53c102d30 2024-03-24T12:45:37.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:36 smithi090 ceph-mon[33646]: pgmap v1789: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:37.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:36 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3504250001' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pool2": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:45:37.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:36 smithi006 ceph-mon[30108]: pgmap v1789: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:37.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:36 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3504250001' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pool2": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:45:37.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:36 smithi006 ceph-mon[41734]: pgmap v1789: 188 pgs: 188 active+clean; 871 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-24T12:45:37.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:36 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3504250001' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pool2": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:45:37.293 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.296+0000 7fd541de2640 1 -- 172.21.15.6:0/809583616 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pool2": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd53c04efc0 con 0x7fd53c102d30 2024-03-24T12:45:37.892 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.895+0000 7fd5397fa640 1 -- 172.21.15.6:0/809583616 <== mon.2 v2:172.21.15.6:3301/0 7 ==== osd_map(793..793 src has 255..793) v4 ==== 296+0+0 (secure 0 0 0) 0x7fd524003e10 con 0x7fd53c102d30 2024-03-24T12:45:37.892 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.895+0000 7fd5397fa640 1 -- 172.21.15.6:0/809583616 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=794}) v3 -- 0x7fd528079ac0 con 0x7fd53c102d30 2024-03-24T12:45:37.902 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:37.905+0000 7fd5397fa640 1 -- 172.21.15.6:0/809583616 <== mon.2 v2:172.21.15.6:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pool2": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "yes_i_really_really_mean_it": true}]=0 pool 'f99c589a-562e-40f2-a1ef-11b16b6c29e6' removed v793) v1 ==== 248+0+0 (secure 0 0 0) 0x7fd524043030 con 0x7fd53c102d30 2024-03-24T12:45:38.017 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:38.020+0000 7fd541de2640 1 -- 172.21.15.6:0/809583616 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pool2": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd53c105fe0 con 0x7fd53c102d30 2024-03-24T12:45:38.018 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:38.021+0000 7fd5397fa640 1 -- 172.21.15.6:0/809583616 <== mon.2 v2:172.21.15.6:3301/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pool2": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "yes_i_really_really_mean_it": true}]=0 pool 'f99c589a-562e-40f2-a1ef-11b16b6c29e6' does not exist v793) v1 ==== 255+0+0 (secure 0 0 0) 0x7fd524092100 con 0x7fd53c102d30 2024-03-24T12:45:38.018 INFO:tasks.workunit.client.0.smithi006.stderr:pool 'f99c589a-562e-40f2-a1ef-11b16b6c29e6' does not exist 2024-03-24T12:45:38.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:38.023+0000 7fd541de2640 1 -- 172.21.15.6:0/809583616 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fd52806d5d0 msgr2=0x7fd52806fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:45:38.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:38.023+0000 7fd541de2640 1 --2- 172.21.15.6:0/809583616 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fd52806d5d0 0x7fd52806fa90 secure :-1 s=READY pgs=4135 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009bd0 tx=0x7fd52c005f90 comp rx=0 tx=0).stop 2024-03-24T12:45:38.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:38.023+0000 7fd541de2640 1 -- 172.21.15.6:0/809583616 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd53c102d30 msgr2=0x7fd53c197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-24T12:45:38.020 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:38.023+0000 7fd541de2640 1 --2- 172.21.15.6:0/809583616 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd53c102d30 0x7fd53c197290 secure :-1 s=READY pgs=3115 cs=0 l=1 rev1=1 crypto rx=0x7fd524006010 tx=0x7fd5240042f0 comp rx=0 tx=0).stop 2024-03-24T12:45:38.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:38.024+0000 7fd541de2640 1 -- 172.21.15.6:0/809583616 shutdown_connections 2024-03-24T12:45:38.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:38.024+0000 7fd541de2640 1 --2- 172.21.15.6:0/809583616 >> [v2:172.21.15.6:6800/3865852080,v1:172.21.15.6:6801/3865852080] conn(0x7fd52806d5d0 0x7fd52806fa90 unknown :-1 s=CLOSED pgs=4135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:38.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:38.024+0000 7fd541de2640 1 --2- 172.21.15.6:0/809583616 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd53c10b7e0 0x7fd53c19b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:38.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:38.024+0000 7fd541de2640 1 --2- 172.21.15.6:0/809583616 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd53c102d30 0x7fd53c197290 unknown :-1 s=CLOSED pgs=3115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:38.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:38.024+0000 7fd541de2640 1 --2- 172.21.15.6:0/809583616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd53c102410 0x7fd53c196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-24T12:45:38.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:38.024+0000 7fd541de2640 1 -- 172.21.15.6:0/809583616 >> 172.21.15.6:0/809583616 conn(0x7fd53c076400 msgr2=0x7fd53c1031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-24T12:45:38.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:38.024+0000 7fd541de2640 1 -- 172.21.15.6:0/809583616 shutdown_connections 2024-03-24T12:45:38.021 INFO:tasks.workunit.client.0.smithi006.stderr:2024-03-24T12:45:38.024+0000 7fd541de2640 1 -- 172.21.15.6:0/809583616 wait complete. 2024-03-24T12:45:38.030 INFO:tasks.workunit.client.0.smithi006.stderr:+ echo OK 2024-03-24T12:45:38.031 INFO:tasks.workunit.client.0.smithi006.stdout:OK 2024-03-24T12:45:38.031 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-24T12:45:38.031 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-24T12:45:38.061 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-03-24T12:45:38.061 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-24T12:45:38.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3504250001' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pool2": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "yes_i_really_really_mean_it": true}]': finished 2024-03-24T12:45:38.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:37 smithi090 ceph-mon[33646]: osdmap e792: 8 total, 8 up, 8 in 2024-03-24T12:45:38.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/3504250001' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pool2": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:45:38.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:37 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/809583616' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pool2": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:45:38.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:37 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pool2": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:45:38.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3504250001' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pool2": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "yes_i_really_really_mean_it": true}]': finished 2024-03-24T12:45:38.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:37 smithi006 ceph-mon[30108]: osdmap e792: 8 total, 8 up, 8 in 2024-03-24T12:45:38.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/3504250001' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pool2": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:45:38.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:37 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/809583616' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pool2": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:45:38.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:37 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pool2": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:45:38.207 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3504250001' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pool2": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "yes_i_really_really_mean_it": true}]': finished 2024-03-24T12:45:38.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:37 smithi006 ceph-mon[41734]: osdmap e792: 8 total, 8 up, 8 in 2024-03-24T12:45:38.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/3504250001' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "pool2": "54e008b8-0c65-4b2f-9ae3-d0f6346347d3", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:45:38.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:37 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/809583616' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pool2": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:45:38.208 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:37 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pool2": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:45:38.393 DEBUG:teuthology.parallel:result is None 2024-03-24T12:45:38.393 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-24T12:45:38.420 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-24T12:45:38.420 DEBUG:teuthology.orchestra.run.smithi006:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-03-24T12:45:38.483 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-03-24T12:45:38.483 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-24T12:45:38.493 INFO:tasks.cephadm:Teardown begin 2024-03-24T12:45:38.494 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-24T12:45:38.551 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-24T12:45:38.587 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-24T12:45:38.587 DEBUG:teuthology.orchestra.run.smithi006:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-24T12:45:38.605 DEBUG:teuthology.orchestra.run.smithi090:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-24T12:45:38.645 INFO:tasks.cephadm:Stopping all daemons... 2024-03-24T12:45:38.646 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-24T12:45:38.646 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mon.a 2024-03-24T12:45:38.951 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:38 smithi006 systemd[1]: Stopping Ceph mon.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:45:38.951 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:38 smithi006 ceph-mon[30108]: pgmap v1791: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:45:38.951 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:38 smithi006 ceph-mon[30108]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-24T12:45:38.951 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pool2": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "yes_i_really_really_mean_it": true}]': finished 2024-03-24T12:45:38.951 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:38 smithi006 ceph-mon[30108]: osdmap e793: 8 total, 8 up, 8 in 2024-03-24T12:45:38.952 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:38 smithi006 ceph-mon[30108]: from='client.? 172.21.15.6:0/809583616' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pool2": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:45:38.952 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:38 smithi006 ceph-mon[30108]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pool2": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:45:38.952 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:38 smithi006 ceph-mon[41734]: pgmap v1791: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:45:38.952 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:38 smithi006 ceph-mon[41734]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-24T12:45:38.952 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pool2": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "yes_i_really_really_mean_it": true}]': finished 2024-03-24T12:45:38.952 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:38 smithi006 ceph-mon[41734]: osdmap e793: 8 total, 8 up, 8 in 2024-03-24T12:45:38.952 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:38 smithi006 ceph-mon[41734]: from='client.? 172.21.15.6:0/809583616' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pool2": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:45:38.952 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:38 smithi006 ceph-mon[41734]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pool2": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:45:39.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:38 smithi090 ceph-mon[33646]: pgmap v1791: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-24T12:45:39.174 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:38 smithi090 ceph-mon[33646]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-24T12:45:39.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pool2": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "yes_i_really_really_mean_it": true}]': finished 2024-03-24T12:45:39.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:38 smithi090 ceph-mon[33646]: osdmap e793: 8 total, 8 up, 8 in 2024-03-24T12:45:39.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:38 smithi090 ceph-mon[33646]: from='client.? 172.21.15.6:0/809583616' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pool2": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:45:39.175 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:38 smithi090 ceph-mon[33646]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "pool2": "f99c589a-562e-40f2-a1ef-11b16b6c29e6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-24T12:45:39.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:38 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-a[30085]: 2024-03-24T12:45:38.954+0000 7f5e24d51700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T12:45:39.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:38 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-a[30085]: 2024-03-24T12:45:38.954+0000 7f5e24d51700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-24T12:45:39.207 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:39 smithi006 podman[159513]: 2024-03-24 12:45:39.04758285 +0000 UTC m=+0.213380447 container died dcf24dc5c2fcfa3c3f11afa79ccfc9a7fb55ce39eb84d228ea0714052441be44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T12:45:39.707 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:39 smithi006 podman[159513]: 2024-03-24 12:45:39.528422483 +0000 UTC m=+0.694220088 container cleanup dcf24dc5c2fcfa3c3f11afa79ccfc9a7fb55ce39eb84d228ea0714052441be44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-a, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-24T12:45:39.707 INFO:journalctl@ceph.mon.a.smithi006.stdout:Mar 24 12:45:39 smithi006 bash[159513]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-a 2024-03-24T12:45:39.948 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mon.a.service' 2024-03-24T12:45:39.989 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T12:45:39.990 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-24T12:45:39.990 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-03-24T12:45:39.990 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mon.c 2024-03-24T12:45:40.331 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:40 smithi006 systemd[1]: Stopping Ceph mon.c for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:45:40.663 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:45:40 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:45:40] ENGINE Bus STOPPING 2024-03-24T12:45:40.664 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:45:40 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:45:40] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-24T12:45:40.664 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:45:40 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:45:40] ENGINE Bus STOPPED 2024-03-24T12:45:40.664 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:45:40 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:45:40] ENGINE Bus STARTING 2024-03-24T12:45:40.664 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:45:40 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:45:40] ENGINE Serving on http://:::9283 2024-03-24T12:45:40.664 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:45:40 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y[30362]: [24/Mar/2024:12:45:40] ENGINE Bus STARTED 2024-03-24T12:45:40.664 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:40 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-c[41710]: 2024-03-24T12:45:40.334+0000 7fc7a9ca3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T12:45:40.664 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:40 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-c[41710]: 2024-03-24T12:45:40.334+0000 7fc7a9ca3700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-03-24T12:45:40.664 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:40 smithi006 podman[159645]: 2024-03-24 12:45:40.407766402 +0000 UTC m=+0.183568293 container died a825aff8c08672afdc0086b47810afd59b011af2d0b451b557193b93ea52aba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-c, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-24T12:45:40.957 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:40 smithi006 podman[159645]: 2024-03-24 12:45:40.892226173 +0000 UTC m=+0.668028048 container cleanup a825aff8c08672afdc0086b47810afd59b011af2d0b451b557193b93ea52aba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS) 2024-03-24T12:45:40.957 INFO:journalctl@ceph.mon.c.smithi006.stdout:Mar 24 12:45:40 smithi006 bash[159645]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-c 2024-03-24T12:45:41.258 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mon.c.service' 2024-03-24T12:45:41.295 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T12:45:41.295 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-03-24T12:45:41.295 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-03-24T12:45:41.295 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mon.b 2024-03-24T12:45:41.595 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:41 smithi090 systemd[1]: Stopping Ceph mon.b for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:45:41.924 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:41 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-b[33623]: 2024-03-24T12:45:41.599+0000 7f622450a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T12:45:41.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:41 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-b[33623]: 2024-03-24T12:45:41.599+0000 7f622450a700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-03-24T12:45:41.925 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:41 smithi090 podman[78918]: 2024-03-24 12:45:41.773667311 +0000 UTC m=+0.278626819 container died f1cf4ff3f7c6c54498ea2f5bfe782512af3d8322b2140a50b062fae894a1bf6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T12:45:42.424 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:42 smithi090 podman[78918]: 2024-03-24 12:45:42.28203783 +0000 UTC m=+0.786997337 container cleanup f1cf4ff3f7c6c54498ea2f5bfe782512af3d8322b2140a50b062fae894a1bf6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-b, RELEASE=squid-db0330b, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-24T12:45:42.424 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:42 smithi090 bash[78918]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-b 2024-03-24T12:45:42.703 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mon.b.service' 2024-03-24T12:45:42.727 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:42 smithi090 podman[78937]: 2024-03-24 12:45:42.467070041 +0000 UTC m=+0.694570323 container remove f1cf4ff3f7c6c54498ea2f5bfe782512af3d8322b2140a50b062fae894a1bf6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mon-b, RELEASE=squid-db0330b, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T12:45:42.727 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:42 smithi090 systemd[1]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mon.b.service: Deactivated successfully. 2024-03-24T12:45:42.727 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:42 smithi090 systemd[1]: Stopped Ceph mon.b for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:45:42.727 INFO:journalctl@ceph.mon.b.smithi090.stdout:Mar 24 12:45:42 smithi090 systemd[1]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mon.b.service: Consumed 30.809s CPU time. 2024-03-24T12:45:42.750 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T12:45:42.750 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-03-24T12:45:42.750 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-24T12:45:42.750 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mgr.y 2024-03-24T12:45:43.088 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:45:42 smithi006 systemd[1]: Stopping Ceph mgr.y for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:45:43.443 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:45:43 smithi006 podman[159877]: 2024-03-24 12:45:43.097466534 +0000 UTC m=+0.146669394 container died efc19a5fffa4bc3c518f0109b506f1f2e76337d40b33778a31b691284786b4fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-24T12:45:43.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:45:43 smithi006 podman[159877]: 2024-03-24 12:45:43.530651367 +0000 UTC m=+0.579854222 container cleanup efc19a5fffa4bc3c518f0109b506f1f2e76337d40b33778a31b691284786b4fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=) 2024-03-24T12:45:43.707 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:45:43 smithi006 bash[159877]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y 2024-03-24T12:45:44.094 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mgr.y.service' 2024-03-24T12:45:44.118 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:45:43 smithi006 podman[159896]: 2024-03-24 12:45:43.865593895 +0000 UTC m=+0.767804394 container remove efc19a5fffa4bc3c518f0109b506f1f2e76337d40b33778a31b691284786b4fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-y, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T12:45:44.119 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:45:44 smithi006 systemd[1]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mgr.y.service: Deactivated successfully. 2024-03-24T12:45:44.119 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:45:44 smithi006 systemd[1]: Stopped Ceph mgr.y for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:45:44.119 INFO:journalctl@ceph.mgr.y.smithi006.stdout:Mar 24 12:45:44 smithi006 systemd[1]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mgr.y.service: Consumed 2min 11.663s CPU time. 2024-03-24T12:45:44.132 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T12:45:44.133 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-24T12:45:44.133 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-24T12:45:44.133 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mgr.x 2024-03-24T12:45:44.424 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:45:44 smithi090 systemd[1]: Stopping Ceph mgr.x for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:45:44.751 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:45:44 smithi090 podman[79049]: 2024-03-24 12:45:44.479293486 +0000 UTC m=+0.153583844 container died 4b38178b7a2effbe65b9411baabd5bc9120c8d596ead15978e8d63fdf30d0961 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, io.buildah.version=1.33.5) 2024-03-24T12:45:45.102 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:45:44 smithi090 podman[79049]: 2024-03-24 12:45:44.838655456 +0000 UTC m=+0.512945824 container cleanup 4b38178b7a2effbe65b9411baabd5bc9120c8d596ead15978e8d63fdf30d0961 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-24T12:45:45.102 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Mar 24 12:45:44 smithi090 bash[79049]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-mgr-x 2024-03-24T12:45:45.242 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@mgr.x.service' 2024-03-24T12:45:45.279 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T12:45:45.280 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-24T12:45:45.280 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-24T12:45:45.280 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.0 2024-03-24T12:45:45.600 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:45:45 smithi006 systemd[1]: Stopping Ceph osd.0 for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:45:45.957 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:45:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-0[46125]: 2024-03-24T12:45:45.602+0000 7ff5e875e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T12:45:45.957 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:45:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-0[46125]: 2024-03-24T12:45:45.602+0000 7ff5e875e700 -1 osd.0 793 *** Got signal Terminated *** 2024-03-24T12:45:45.957 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:45:45 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-0[46125]: 2024-03-24T12:45:45.602+0000 7ff5e875e700 -1 osd.0 793 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T12:45:51.249 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:45:50 smithi006 podman[160009]: 2024-03-24 12:45:50.994868318 +0000 UTC m=+5.510634517 container died 45597b65e31ec775757ee9c10b8d09f607c8c138daff77b4a6f7394b9ec71bf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-0, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-24T12:45:51.582 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:45:51 smithi006 podman[160009]: 2024-03-24 12:45:51.343656681 +0000 UTC m=+5.859422891 container cleanup 45597b65e31ec775757ee9c10b8d09f607c8c138daff77b4a6f7394b9ec71bf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-0, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-24T12:45:51.582 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:45:51 smithi006 bash[160009]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-0 2024-03-24T12:45:51.957 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:45:51 smithi006 podman[160042]: 2024-03-24 12:45:51.586750464 +0000 UTC m=+0.587699124 container remove 45597b65e31ec775757ee9c10b8d09f607c8c138daff77b4a6f7394b9ec71bf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-db0330b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-24T12:45:52.268 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:45:52 smithi006 podman[160141]: 2024-03-24 12:45:51.908859839 +0000 UTC m=+0.018789959 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-24T12:45:52.268 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:45:52 smithi006 podman[160141]: 2024-03-24 12:45:52.046275177 +0000 UTC m=+0.156205274 container create a5a43ebd2c0041b59198a96e53fdb907c4a9ee8a7d2684247bbbbf202f4afae8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-24T12:45:52.572 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:45:52 smithi006 podman[160141]: 2024-03-24 12:45:52.273380924 +0000 UTC m=+0.383311034 container init a5a43ebd2c0041b59198a96e53fdb907c4a9ee8a7d2684247bbbbf202f4afae8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-24T12:45:52.572 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:45:52 smithi006 podman[160141]: 2024-03-24 12:45:52.278695255 +0000 UTC m=+0.388625358 container start a5a43ebd2c0041b59198a96e53fdb907c4a9ee8a7d2684247bbbbf202f4afae8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-24T12:45:52.572 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:45:52 smithi006 podman[160141]: 2024-03-24 12:45:52.356689279 +0000 UTC m=+0.466619384 container attach a5a43ebd2c0041b59198a96e53fdb907c4a9ee8a7d2684247bbbbf202f4afae8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-24T12:45:52.854 INFO:journalctl@ceph.osd.0.smithi006.stdout:Mar 24 12:45:52 smithi006 podman[160141]: 2024-03-24 12:45:52.57754519 +0000 UTC m=+0.687475284 container died a5a43ebd2c0041b59198a96e53fdb907c4a9ee8a7d2684247bbbbf202f4afae8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T12:45:53.166 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.0.service' 2024-03-24T12:45:53.210 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T12:45:53.211 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-24T12:45:53.211 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-24T12:45:53.211 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.1 2024-03-24T12:45:53.457 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:45:53 smithi006 systemd[1]: Stopping Ceph osd.1 for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:45:53.957 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:45:53 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-1[52194]: 2024-03-24T12:45:53.607+0000 7fa94c84f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T12:45:53.957 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:45:53 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-1[52194]: 2024-03-24T12:45:53.607+0000 7fa94c84f700 -1 osd.1 793 *** Got signal Terminated *** 2024-03-24T12:45:53.957 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:45:53 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-1[52194]: 2024-03-24T12:45:53.607+0000 7fa94c84f700 -1 osd.1 793 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T12:45:59.335 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:45:59 smithi006 podman[160274]: 2024-03-24 12:45:59.028910403 +0000 UTC m=+5.587362942 container died 1873bfd1b9888dbf594b5e14853d7e708a169d17f96fefc7323d167753c39838 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=squid-db0330b, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T12:45:59.593 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:45:59 smithi006 podman[160274]: 2024-03-24 12:45:59.430849925 +0000 UTC m=+5.989302463 container cleanup 1873bfd1b9888dbf594b5e14853d7e708a169d17f96fefc7323d167753c39838 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-1, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-24T12:45:59.594 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:45:59 smithi006 bash[160274]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-1 2024-03-24T12:45:59.957 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:45:59 smithi006 podman[160296]: 2024-03-24 12:45:59.598624695 +0000 UTC m=+0.559307072 container remove 1873bfd1b9888dbf594b5e14853d7e708a169d17f96fefc7323d167753c39838 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-1, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T12:46:00.296 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:46:00 smithi006 podman[160394]: 2024-03-24 12:45:59.910335308 +0000 UTC m=+0.017502495 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-24T12:46:00.297 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:46:00 smithi006 podman[160394]: 2024-03-24 12:46:00.016151757 +0000 UTC m=+0.123318928 container create 8abea9968936e7b8087c322375b667f794d8c77c9e8a0513acba39bdc62d7295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T12:46:00.297 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:46:00 smithi006 podman[160394]: 2024-03-24 12:46:00.218440044 +0000 UTC m=+0.325607216 container init 8abea9968936e7b8087c322375b667f794d8c77c9e8a0513acba39bdc62d7295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-24T12:46:00.297 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:46:00 smithi006 podman[160394]: 2024-03-24 12:46:00.224215246 +0000 UTC m=+0.331382418 container start 8abea9968936e7b8087c322375b667f794d8c77c9e8a0513acba39bdc62d7295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-24T12:46:00.297 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:46:00 smithi006 podman[160394]: 2024-03-24 12:46:00.301622213 +0000 UTC m=+0.408789397 container attach 8abea9968936e7b8087c322375b667f794d8c77c9e8a0513acba39bdc62d7295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-24T12:46:00.707 INFO:journalctl@ceph.osd.1.smithi006.stdout:Mar 24 12:46:00 smithi006 podman[160394]: 2024-03-24 12:46:00.529586051 +0000 UTC m=+0.636753236 container died 8abea9968936e7b8087c322375b667f794d8c77c9e8a0513acba39bdc62d7295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T12:46:01.103 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.1.service' 2024-03-24T12:46:01.147 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T12:46:01.147 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-24T12:46:01.147 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-24T12:46:01.147 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.2 2024-03-24T12:46:01.457 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:46:01 smithi006 systemd[1]: Stopping Ceph osd.2 for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:46:01.957 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:46:01 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-2[58188]: 2024-03-24T12:46:01.485+0000 7f5fa2178700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T12:46:01.957 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:46:01 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-2[58188]: 2024-03-24T12:46:01.485+0000 7f5fa2178700 -1 osd.2 793 *** Got signal Terminated *** 2024-03-24T12:46:01.957 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:46:01 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-2[58188]: 2024-03-24T12:46:01.485+0000 7f5fa2178700 -1 osd.2 793 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T12:46:07.238 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:46:06 smithi006 podman[160525]: 2024-03-24 12:46:06.970401458 +0000 UTC m=+5.593215470 container died e9be3688c93a0f5ce549f34b50b582adc7d7526e6c28c478862a54413d9e4b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-24T12:46:07.595 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:46:07 smithi006 podman[160525]: 2024-03-24 12:46:07.332761158 +0000 UTC m=+5.955575169 container cleanup e9be3688c93a0f5ce549f34b50b582adc7d7526e6c28c478862a54413d9e4b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-24T12:46:07.595 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:46:07 smithi006 bash[160525]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-2 2024-03-24T12:46:07.929 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:46:07 smithi006 podman[160546]: 2024-03-24 12:46:07.600491344 +0000 UTC m=+0.630633094 container remove e9be3688c93a0f5ce549f34b50b582adc7d7526e6c28c478862a54413d9e4b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux ) 2024-03-24T12:46:08.207 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:46:07 smithi006 podman[160645]: 2024-03-24 12:46:07.934666123 +0000 UTC m=+0.097957829 container create 91593aaa106aabf0cef09b03becd00c485b51701a4f8553ba04687c1deeeb340 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True) 2024-03-24T12:46:08.207 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:46:07 smithi006 podman[160645]: 2024-03-24 12:46:07.854063411 +0000 UTC m=+0.017355083 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-24T12:46:08.207 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:46:08 smithi006 podman[160645]: 2024-03-24 12:46:08.153974869 +0000 UTC m=+0.317266545 container init 91593aaa106aabf0cef09b03becd00c485b51701a4f8553ba04687c1deeeb340 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307) 2024-03-24T12:46:08.208 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:46:08 smithi006 podman[160645]: 2024-03-24 12:46:08.159358969 +0000 UTC m=+0.322650634 container start 91593aaa106aabf0cef09b03becd00c485b51701a4f8553ba04687c1deeeb340 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T12:46:08.707 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:46:08 smithi006 podman[160645]: 2024-03-24 12:46:08.245506658 +0000 UTC m=+0.408798345 container attach 91593aaa106aabf0cef09b03becd00c485b51701a4f8553ba04687c1deeeb340 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-24T12:46:08.707 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:46:08 smithi006 podman[160645]: 2024-03-24 12:46:08.456501939 +0000 UTC m=+0.619793603 container died 91593aaa106aabf0cef09b03becd00c485b51701a4f8553ba04687c1deeeb340 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-24T12:46:08.995 INFO:journalctl@ceph.osd.2.smithi006.stdout:Mar 24 12:46:08 smithi006 podman[160691]: 2024-03-24 12:46:08.982364686 +0000 UTC m=+0.521310867 container remove 91593aaa106aabf0cef09b03becd00c485b51701a4f8553ba04687c1deeeb340 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-24T12:46:09.001 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.2.service' 2024-03-24T12:46:09.044 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T12:46:09.044 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-24T12:46:09.044 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-24T12:46:09.045 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.3 2024-03-24T12:46:09.366 INFO:journalctl@ceph.osd.3.smithi006.stdout:Mar 24 12:46:09 smithi006 systemd[1]: Stopping Ceph osd.3 for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:46:09.707 INFO:journalctl@ceph.osd.3.smithi006.stdout:Mar 24 12:46:09 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-3[64519]: 2024-03-24T12:46:09.370+0000 7f30b2abb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T12:46:09.707 INFO:journalctl@ceph.osd.3.smithi006.stdout:Mar 24 12:46:09 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-3[64519]: 2024-03-24T12:46:09.370+0000 7f30b2abb700 -1 osd.3 793 *** Got signal Terminated *** 2024-03-24T12:46:09.707 INFO:journalctl@ceph.osd.3.smithi006.stdout:Mar 24 12:46:09 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-3[64519]: 2024-03-24T12:46:09.370+0000 7f30b2abb700 -1 osd.3 793 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T12:46:13.347 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:13 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:13.208+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.969360+0000 front 2024-03-24T12:45:47.969453+0000 (oldest deadline 2024-03-24T12:46:13.169951+0000) 2024-03-24T12:46:13.674 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:13 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4[39167]: 2024-03-24T12:46:13.351+0000 7fb7df3dc700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.081869+0000 front 2024-03-24T12:45:47.082005+0000 (oldest deadline 2024-03-24T12:46:12.981736+0000) 2024-03-24T12:46:14.457 INFO:journalctl@ceph.osd.3.smithi006.stdout:Mar 24 12:46:13 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-3[64519]: 2024-03-24T12:46:13.975+0000 7f30ab0a7700 -1 osd.3 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.453299+0000 front 2024-03-24T12:45:47.453067+0000 (oldest deadline 2024-03-24T12:46:13.353275+0000) 2024-03-24T12:46:14.674 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:14 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4[39167]: 2024-03-24T12:46:14.384+0000 7fb7df3dc700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.081869+0000 front 2024-03-24T12:45:47.082005+0000 (oldest deadline 2024-03-24T12:46:12.981736+0000) 2024-03-24T12:46:14.675 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:14 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:14.241+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.969360+0000 front 2024-03-24T12:45:47.969453+0000 (oldest deadline 2024-03-24T12:46:13.169951+0000) 2024-03-24T12:46:14.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:14 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:14.295+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:15.006 INFO:journalctl@ceph.osd.3.smithi006.stdout:Mar 24 12:46:14 smithi006 podman[160776]: 2024-03-24 12:46:14.746345351 +0000 UTC m=+5.475215472 container died e9a8f31420fd76e1c2268e86e0e2b7cc30692c3d44fdacb13245af5d3aea44ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-24T12:46:15.372 INFO:journalctl@ceph.osd.3.smithi006.stdout:Mar 24 12:46:15 smithi006 podman[160776]: 2024-03-24 12:46:15.118349658 +0000 UTC m=+5.847219770 container cleanup e9a8f31420fd76e1c2268e86e0e2b7cc30692c3d44fdacb13245af5d3aea44ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-3, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, ceph=True) 2024-03-24T12:46:15.373 INFO:journalctl@ceph.osd.3.smithi006.stdout:Mar 24 12:46:15 smithi006 bash[160776]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-3 2024-03-24T12:46:15.674 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:15 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4[39167]: 2024-03-24T12:46:15.399+0000 7fb7df3dc700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.081869+0000 front 2024-03-24T12:45:47.082005+0000 (oldest deadline 2024-03-24T12:46:12.981736+0000) 2024-03-24T12:46:15.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:15 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:15.250+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:15.675 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:15 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:15.197+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.969360+0000 front 2024-03-24T12:45:47.969453+0000 (oldest deadline 2024-03-24T12:46:13.169951+0000) 2024-03-24T12:46:15.675 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:15 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:15.292+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:15.707 INFO:journalctl@ceph.osd.3.smithi006.stdout:Mar 24 12:46:15 smithi006 podman[160799]: 2024-03-24 12:46:15.377718225 +0000 UTC m=+0.629894371 container remove e9a8f31420fd76e1c2268e86e0e2b7cc30692c3d44fdacb13245af5d3aea44ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-24T12:46:16.034 INFO:journalctl@ceph.osd.3.smithi006.stdout:Mar 24 12:46:15 smithi006 podman[160896]: 2024-03-24 12:46:15.728745187 +0000 UTC m=+0.111966918 container create 63eef48e33743a40a5d2e2f4d32b71482a996494c181b97768918311e86c5469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-24T12:46:16.034 INFO:journalctl@ceph.osd.3.smithi006.stdout:Mar 24 12:46:15 smithi006 podman[160896]: 2024-03-24 12:46:15.633821312 +0000 UTC m=+0.017043009 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-24T12:46:16.034 INFO:journalctl@ceph.osd.3.smithi006.stdout:Mar 24 12:46:15 smithi006 podman[160896]: 2024-03-24 12:46:15.922753882 +0000 UTC m=+0.305975574 container init 63eef48e33743a40a5d2e2f4d32b71482a996494c181b97768918311e86c5469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T12:46:16.034 INFO:journalctl@ceph.osd.3.smithi006.stdout:Mar 24 12:46:15 smithi006 podman[160896]: 2024-03-24 12:46:15.928483997 +0000 UTC m=+0.311705693 container start 63eef48e33743a40a5d2e2f4d32b71482a996494c181b97768918311e86c5469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-24T12:46:16.035 INFO:journalctl@ceph.osd.3.smithi006.stdout:Mar 24 12:46:16 smithi006 podman[160896]: 2024-03-24 12:46:16.039071001 +0000 UTC m=+0.422292702 container attach 63eef48e33743a40a5d2e2f4d32b71482a996494c181b97768918311e86c5469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-24T12:46:16.457 INFO:journalctl@ceph.osd.3.smithi006.stdout:Mar 24 12:46:16 smithi006 podman[160896]: 2024-03-24 12:46:16.226029906 +0000 UTC m=+0.609251596 container died 63eef48e33743a40a5d2e2f4d32b71482a996494c181b97768918311e86c5469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-24T12:46:16.674 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:16 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4[39167]: 2024-03-24T12:46:16.352+0000 7fb7df3dc700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.081869+0000 front 2024-03-24T12:45:47.082005+0000 (oldest deadline 2024-03-24T12:46:12.981736+0000) 2024-03-24T12:46:16.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:16 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:16.248+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:16.675 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:16 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:16.188+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.969360+0000 front 2024-03-24T12:45:47.969453+0000 (oldest deadline 2024-03-24T12:46:13.169951+0000) 2024-03-24T12:46:16.675 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:16 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:16.316+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:16.844 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.3.service' 2024-03-24T12:46:16.885 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T12:46:16.885 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-24T12:46:16.885 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-24T12:46:16.885 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.4 2024-03-24T12:46:17.174 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:16 smithi090 systemd[1]: Stopping Ceph osd.4 for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:46:17.674 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:17 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:17.209+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.969360+0000 front 2024-03-24T12:45:47.969453+0000 (oldest deadline 2024-03-24T12:46:13.169951+0000) 2024-03-24T12:46:17.675 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:17 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:17.326+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:17.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:17 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:17.284+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:17.675 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:17 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4[39167]: 2024-03-24T12:46:17.189+0000 7fb7e2de8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T12:46:17.675 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:17 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4[39167]: 2024-03-24T12:46:17.189+0000 7fb7e2de8700 -1 osd.4 793 *** Got signal Terminated *** 2024-03-24T12:46:17.675 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:17 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4[39167]: 2024-03-24T12:46:17.189+0000 7fb7e2de8700 -1 osd.4 793 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T12:46:17.676 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:17 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4[39167]: 2024-03-24T12:46:17.316+0000 7fb7df3dc700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.081869+0000 front 2024-03-24T12:45:47.082005+0000 (oldest deadline 2024-03-24T12:46:12.981736+0000) 2024-03-24T12:46:18.674 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:18 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4[39167]: 2024-03-24T12:46:18.319+0000 7fb7df3dc700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.081869+0000 front 2024-03-24T12:45:47.082005+0000 (oldest deadline 2024-03-24T12:46:12.981736+0000) 2024-03-24T12:46:18.675 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:18 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:18.318+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:18.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:18 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:18.291+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:18.675 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:18 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:18.203+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.969360+0000 front 2024-03-24T12:45:47.969453+0000 (oldest deadline 2024-03-24T12:46:13.169951+0000) 2024-03-24T12:46:19.425 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:19 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:19.175+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.969360+0000 front 2024-03-24T12:45:47.969453+0000 (oldest deadline 2024-03-24T12:46:13.169951+0000) 2024-03-24T12:46:19.425 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:19 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:19.335+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:19.425 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:19 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:19.261+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:19.426 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:19 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4[39167]: 2024-03-24T12:46:19.368+0000 7fb7df3dc700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.081869+0000 front 2024-03-24T12:45:47.082005+0000 (oldest deadline 2024-03-24T12:46:12.981736+0000) 2024-03-24T12:46:19.426 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:19 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4[39167]: 2024-03-24T12:46:19.368+0000 7fb7df3dc700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:55.783186+0000 front 2024-03-24T12:45:55.783276+0000 (oldest deadline 2024-03-24T12:46:19.283105+0000) 2024-03-24T12:46:20.424 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:20 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:20.308+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:20.425 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:20 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:20.130+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.969360+0000 front 2024-03-24T12:45:47.969453+0000 (oldest deadline 2024-03-24T12:46:13.169951+0000) 2024-03-24T12:46:20.425 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:20 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:20.130+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:57.870891+0000 front 2024-03-24T12:45:57.871048+0000 (oldest deadline 2024-03-24T12:46:19.570860+0000) 2024-03-24T12:46:20.425 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:20 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:20.355+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:20.425 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:20 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4[39167]: 2024-03-24T12:46:20.326+0000 7fb7df3dc700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.081869+0000 front 2024-03-24T12:45:47.082005+0000 (oldest deadline 2024-03-24T12:46:12.981736+0000) 2024-03-24T12:46:20.425 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:20 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4[39167]: 2024-03-24T12:46:20.326+0000 7fb7df3dc700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:55.783186+0000 front 2024-03-24T12:45:55.783276+0000 (oldest deadline 2024-03-24T12:46:19.283105+0000) 2024-03-24T12:46:21.674 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:21 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:21.344+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:21.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:21 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:21.344+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:21.675 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:21 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:21.179+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.969360+0000 front 2024-03-24T12:45:47.969453+0000 (oldest deadline 2024-03-24T12:46:13.169951+0000) 2024-03-24T12:46:21.675 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:21 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:21.179+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:57.870891+0000 front 2024-03-24T12:45:57.871048+0000 (oldest deadline 2024-03-24T12:46:19.570860+0000) 2024-03-24T12:46:21.675 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:21 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:21.349+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:21.675 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:21 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:21.349+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.652310+0000 front 2024-03-24T12:45:54.652377+0000 (oldest deadline 2024-03-24T12:46:20.552263+0000) 2024-03-24T12:46:21.676 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:21 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4[39167]: 2024-03-24T12:46:21.290+0000 7fb7df3dc700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.081869+0000 front 2024-03-24T12:45:47.082005+0000 (oldest deadline 2024-03-24T12:46:12.981736+0000) 2024-03-24T12:46:21.676 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:21 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4[39167]: 2024-03-24T12:46:21.290+0000 7fb7df3dc700 -1 osd.4 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:55.783186+0000 front 2024-03-24T12:45:55.783276+0000 (oldest deadline 2024-03-24T12:46:19.283105+0000) 2024-03-24T12:46:22.674 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:22.390+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:22.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:22.390+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:22.675 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:22.197+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.969360+0000 front 2024-03-24T12:45:47.969453+0000 (oldest deadline 2024-03-24T12:46:13.169951+0000) 2024-03-24T12:46:22.675 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:22.197+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:57.870891+0000 front 2024-03-24T12:45:57.871048+0000 (oldest deadline 2024-03-24T12:46:19.570860+0000) 2024-03-24T12:46:22.675 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:22.318+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:22.675 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:22 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:22.318+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.652310+0000 front 2024-03-24T12:45:54.652377+0000 (oldest deadline 2024-03-24T12:46:20.552263+0000) 2024-03-24T12:46:22.977 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:22 smithi090 podman[79200]: 2024-03-24 12:46:22.71250642 +0000 UTC m=+5.628452504 container died ed148b410df5fe7c5c68291c0f9826954f90814e4eff6b5ff3080d9cee9311ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, ceph=True) 2024-03-24T12:46:23.340 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:23 smithi090 podman[79200]: 2024-03-24 12:46:23.080423859 +0000 UTC m=+5.996369942 container cleanup ed148b410df5fe7c5c68291c0f9826954f90814e4eff6b5ff3080d9cee9311ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-24T12:46:23.340 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:23 smithi090 bash[79200]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4 2024-03-24T12:46:23.340 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:23.173+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.969360+0000 front 2024-03-24T12:45:47.969453+0000 (oldest deadline 2024-03-24T12:46:13.169951+0000) 2024-03-24T12:46:23.340 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:23.173+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:57.870891+0000 front 2024-03-24T12:45:57.871048+0000 (oldest deadline 2024-03-24T12:46:19.570860+0000) 2024-03-24T12:46:23.674 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:23 smithi090 podman[79223]: 2024-03-24 12:46:23.382353978 +0000 UTC m=+0.667348059 container remove ed148b410df5fe7c5c68291c0f9826954f90814e4eff6b5ff3080d9cee9311ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-24T12:46:23.675 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:23.344+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:23.675 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:23.344+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.652310+0000 front 2024-03-24T12:45:54.652377+0000 (oldest deadline 2024-03-24T12:46:20.552263+0000) 2024-03-24T12:46:23.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:23.344+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:23.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:23 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:23.344+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:24.000 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:23 smithi090 podman[79322]: 2024-03-24 12:46:23.644664659 +0000 UTC m=+0.017891487 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-24T12:46:24.000 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:23 smithi090 podman[79322]: 2024-03-24 12:46:23.767112861 +0000 UTC m=+0.140339728 container create bcd6a668a2cf7f90ae9b7cc696305807bda319422a75f3cc54ca1160f80dad32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-24T12:46:24.299 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:24.199+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.969360+0000 front 2024-03-24T12:45:47.969453+0000 (oldest deadline 2024-03-24T12:46:13.169951+0000) 2024-03-24T12:46:24.299 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:24.199+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:57.870891+0000 front 2024-03-24T12:45:57.871048+0000 (oldest deadline 2024-03-24T12:46:19.570860+0000) 2024-03-24T12:46:24.299 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:24 smithi090 podman[79322]: 2024-03-24 12:46:24.005066478 +0000 UTC m=+0.378293305 container init bcd6a668a2cf7f90ae9b7cc696305807bda319422a75f3cc54ca1160f80dad32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b) 2024-03-24T12:46:24.299 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:24 smithi090 podman[79322]: 2024-03-24 12:46:24.010780681 +0000 UTC m=+0.384007509 container start bcd6a668a2cf7f90ae9b7cc696305807bda319422a75f3cc54ca1160f80dad32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-24T12:46:24.299 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:24 smithi090 podman[79322]: 2024-03-24 12:46:24.087724803 +0000 UTC m=+0.460951633 container attach bcd6a668a2cf7f90ae9b7cc696305807bda319422a75f3cc54ca1160f80dad32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-24T12:46:24.586 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:24.321+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:24.586 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:24.321+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:24.586 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:24.353+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:24.586 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:24 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:24.353+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.652310+0000 front 2024-03-24T12:45:54.652377+0000 (oldest deadline 2024-03-24T12:46:20.552263+0000) 2024-03-24T12:46:24.587 INFO:journalctl@ceph.osd.4.smithi090.stdout:Mar 24 12:46:24 smithi090 podman[79322]: 2024-03-24 12:46:24.304534379 +0000 UTC m=+0.677761211 container died bcd6a668a2cf7f90ae9b7cc696305807bda319422a75f3cc54ca1160f80dad32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T12:46:24.859 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.4.service' 2024-03-24T12:46:24.901 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T12:46:24.901 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-24T12:46:24.902 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-24T12:46:24.902 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.5 2024-03-24T12:46:25.174 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:24 smithi090 systemd[1]: Stopping Ceph osd.5 for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:46:25.674 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:25 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:25.347+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:25.674 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:25 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:25.347+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.652310+0000 front 2024-03-24T12:45:54.652377+0000 (oldest deadline 2024-03-24T12:46:20.552263+0000) 2024-03-24T12:46:25.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:25 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:25.291+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:25.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:25 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:25.291+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:25.675 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:25 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:25.225+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.969360+0000 front 2024-03-24T12:45:47.969453+0000 (oldest deadline 2024-03-24T12:46:13.169951+0000) 2024-03-24T12:46:25.675 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:25 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:25.225+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:57.870891+0000 front 2024-03-24T12:45:57.871048+0000 (oldest deadline 2024-03-24T12:46:19.570860+0000) 2024-03-24T12:46:25.675 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:25 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:25.238+0000 7fdb06415700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T12:46:25.675 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:25 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:25.238+0000 7fdb06415700 -1 osd.5 793 *** Got signal Terminated *** 2024-03-24T12:46:25.675 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:25 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:25.238+0000 7fdb06415700 -1 osd.5 793 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T12:46:26.675 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:26 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:26.200+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.969360+0000 front 2024-03-24T12:45:47.969453+0000 (oldest deadline 2024-03-24T12:46:13.169951+0000) 2024-03-24T12:46:26.675 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:26 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:26.200+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:57.870891+0000 front 2024-03-24T12:45:57.871048+0000 (oldest deadline 2024-03-24T12:46:19.570860+0000) 2024-03-24T12:46:26.675 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:26 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:26.352+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:26.675 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:26 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:26.352+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.652310+0000 front 2024-03-24T12:45:54.652377+0000 (oldest deadline 2024-03-24T12:46:20.552263+0000) 2024-03-24T12:46:26.676 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:26 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:26.283+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:26.676 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:26 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:26.283+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:27.675 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:27 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:27.238+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.969360+0000 front 2024-03-24T12:45:47.969453+0000 (oldest deadline 2024-03-24T12:46:13.169951+0000) 2024-03-24T12:46:27.675 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:27 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:27.238+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:57.870891+0000 front 2024-03-24T12:45:57.871048+0000 (oldest deadline 2024-03-24T12:46:19.570860+0000) 2024-03-24T12:46:27.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:27 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:27.262+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:27.676 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:27 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:27.262+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:27.676 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:27 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:27.398+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:27.676 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:27 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:27.398+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.652310+0000 front 2024-03-24T12:45:54.652377+0000 (oldest deadline 2024-03-24T12:46:20.552263+0000) 2024-03-24T12:46:28.674 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:28 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:28.259+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.969360+0000 front 2024-03-24T12:45:47.969453+0000 (oldest deadline 2024-03-24T12:46:13.169951+0000) 2024-03-24T12:46:28.675 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:28 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:28.259+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:57.870891+0000 front 2024-03-24T12:45:57.871048+0000 (oldest deadline 2024-03-24T12:46:19.570860+0000) 2024-03-24T12:46:28.675 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:28 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:28.352+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:28.675 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:28 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:28.352+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.652310+0000 front 2024-03-24T12:45:54.652377+0000 (oldest deadline 2024-03-24T12:46:20.552263+0000) 2024-03-24T12:46:28.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:28 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:28.298+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:28.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:28 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:28.298+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:29.674 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:29 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:29.367+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:29.675 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:29 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:29.367+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.652310+0000 front 2024-03-24T12:45:54.652377+0000 (oldest deadline 2024-03-24T12:46:20.552263+0000) 2024-03-24T12:46:29.675 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:29 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:29.367+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:05.853303+0000 front 2024-03-24T12:46:05.853223+0000 (oldest deadline 2024-03-24T12:46:28.653656+0000) 2024-03-24T12:46:29.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:29 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:29.329+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:29.676 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:29 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:29.329+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:29.676 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:29 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:29.329+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:04.588320+0000 front 2024-03-24T12:46:04.588417+0000 (oldest deadline 2024-03-24T12:46:28.688254+0000) 2024-03-24T12:46:29.676 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:29 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:29.289+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:47.969360+0000 front 2024-03-24T12:45:47.969453+0000 (oldest deadline 2024-03-24T12:46:13.169951+0000) 2024-03-24T12:46:29.676 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:29 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:29.289+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:57.870891+0000 front 2024-03-24T12:45:57.871048+0000 (oldest deadline 2024-03-24T12:46:19.570860+0000) 2024-03-24T12:46:29.676 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:29 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5[45057]: 2024-03-24T12:46:29.289+0000 7fdafea01700 -1 osd.5 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:06.472710+0000 front 2024-03-24T12:46:06.472628+0000 (oldest deadline 2024-03-24T12:46:28.772558+0000) 2024-03-24T12:46:30.675 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:30 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:30.324+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:30.675 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:30 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:30.324+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.652310+0000 front 2024-03-24T12:45:54.652377+0000 (oldest deadline 2024-03-24T12:46:20.552263+0000) 2024-03-24T12:46:30.675 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:30 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:30.324+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:05.853303+0000 front 2024-03-24T12:46:05.853223+0000 (oldest deadline 2024-03-24T12:46:28.653656+0000) 2024-03-24T12:46:30.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:30 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:30.379+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:30.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:30 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:30.379+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:30.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:30 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:30.379+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:04.588320+0000 front 2024-03-24T12:46:04.588417+0000 (oldest deadline 2024-03-24T12:46:28.688254+0000) 2024-03-24T12:46:30.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:30 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:30.379+0000 7fb22c30f700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24430.0:2183 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-03-24T12:46:31.008 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:30 smithi090 podman[79454]: 2024-03-24 12:46:30.741115581 +0000 UTC m=+5.619366032 container died 70bbd0a1a03581fd0a03042d022ef4756a6713d0c7544e2967a44d1d93bc15fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-24T12:46:31.282 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:31 smithi090 podman[79454]: 2024-03-24 12:46:31.120755948 +0000 UTC m=+5.999006397 container cleanup 70bbd0a1a03581fd0a03042d022ef4756a6713d0c7544e2967a44d1d93bc15fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, org.label-schema.license=GPLv2) 2024-03-24T12:46:31.283 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:31 smithi090 bash[79454]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5 2024-03-24T12:46:31.674 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:31 smithi090 podman[79475]: 2024-03-24 12:46:31.415156561 +0000 UTC m=+0.671849557 container remove 70bbd0a1a03581fd0a03042d022ef4756a6713d0c7544e2967a44d1d93bc15fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307) 2024-03-24T12:46:31.675 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:31 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:31.286+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:31.675 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:31 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:31.286+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.652310+0000 front 2024-03-24T12:45:54.652377+0000 (oldest deadline 2024-03-24T12:46:20.552263+0000) 2024-03-24T12:46:31.675 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:31 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:31.286+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:05.853303+0000 front 2024-03-24T12:46:05.853223+0000 (oldest deadline 2024-03-24T12:46:28.653656+0000) 2024-03-24T12:46:31.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:31 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:31.405+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:31.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:31 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:31.405+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:31.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:31 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:31.405+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:04.588320+0000 front 2024-03-24T12:46:04.588417+0000 (oldest deadline 2024-03-24T12:46:28.688254+0000) 2024-03-24T12:46:31.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:31 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:31.405+0000 7fb22c30f700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24430.0:2183 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-03-24T12:46:32.055 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:31 smithi090 podman[79570]: 2024-03-24 12:46:31.642066687 +0000 UTC m=+0.019152663 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-24T12:46:32.056 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:31 smithi090 podman[79570]: 2024-03-24 12:46:31.775006638 +0000 UTC m=+0.152092638 container create ac59ae343d19af3fdce3d04f1aaa90cf914782bcfed818f086c264ab2c472020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0) 2024-03-24T12:46:32.056 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:31 smithi090 podman[79570]: 2024-03-24 12:46:31.977505257 +0000 UTC m=+0.354591211 container init ac59ae343d19af3fdce3d04f1aaa90cf914782bcfed818f086c264ab2c472020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5-deactivate, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T12:46:32.056 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:31 smithi090 podman[79570]: 2024-03-24 12:46:31.983438249 +0000 UTC m=+0.360524204 container start ac59ae343d19af3fdce3d04f1aaa90cf914782bcfed818f086c264ab2c472020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-24T12:46:32.056 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:32 smithi090 podman[79570]: 2024-03-24 12:46:32.060842867 +0000 UTC m=+0.437928840 container attach ac59ae343d19af3fdce3d04f1aaa90cf914782bcfed818f086c264ab2c472020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-24T12:46:32.424 INFO:journalctl@ceph.osd.5.smithi090.stdout:Mar 24 12:46:32 smithi090 podman[79570]: 2024-03-24 12:46:32.271488117 +0000 UTC m=+0.648574066 container died ac59ae343d19af3fdce3d04f1aaa90cf914782bcfed818f086c264ab2c472020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T12:46:32.425 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:32.276+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:32.425 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:32.276+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.652310+0000 front 2024-03-24T12:45:54.652377+0000 (oldest deadline 2024-03-24T12:46:20.552263+0000) 2024-03-24T12:46:32.425 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:32.276+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:05.853303+0000 front 2024-03-24T12:46:05.853223+0000 (oldest deadline 2024-03-24T12:46:28.653656+0000) 2024-03-24T12:46:32.708 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:32.437+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:32.709 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:32.437+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:32.709 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:32.437+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:04.588320+0000 front 2024-03-24T12:46:04.588417+0000 (oldest deadline 2024-03-24T12:46:28.688254+0000) 2024-03-24T12:46:32.709 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:32 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:32.437+0000 7fb22c30f700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24430.0:2183 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-03-24T12:46:32.842 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.5.service' 2024-03-24T12:46:32.884 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T12:46:32.884 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-24T12:46:32.884 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-24T12:46:32.884 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.6 2024-03-24T12:46:33.242 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:32 smithi090 systemd[1]: Stopping Ceph osd.6 for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:46:33.502 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:33.459+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:33.503 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:33.459+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:33.503 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:33.459+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:04.588320+0000 front 2024-03-24T12:46:04.588417+0000 (oldest deadline 2024-03-24T12:46:28.688254+0000) 2024-03-24T12:46:33.503 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:33.459+0000 7fb22c30f700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24430.0:2183 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-03-24T12:46:33.503 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:33.246+0000 7f1d4b781700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T12:46:33.503 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:33.246+0000 7f1d4b781700 -1 osd.6 793 *** Got signal Terminated *** 2024-03-24T12:46:33.503 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:33.246+0000 7f1d4b781700 -1 osd.6 793 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T12:46:33.503 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:33.266+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:33.503 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:33.266+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.652310+0000 front 2024-03-24T12:45:54.652377+0000 (oldest deadline 2024-03-24T12:46:20.552263+0000) 2024-03-24T12:46:33.503 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:33 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:33.266+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:05.853303+0000 front 2024-03-24T12:46:05.853223+0000 (oldest deadline 2024-03-24T12:46:28.653656+0000) 2024-03-24T12:46:34.488 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:34 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:34.234+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:34.488 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:34 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:34.234+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.652310+0000 front 2024-03-24T12:45:54.652377+0000 (oldest deadline 2024-03-24T12:46:20.552263+0000) 2024-03-24T12:46:34.488 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:34 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:34.234+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:05.853303+0000 front 2024-03-24T12:46:05.853223+0000 (oldest deadline 2024-03-24T12:46:28.653656+0000) 2024-03-24T12:46:34.924 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:34 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:34.491+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:34.924 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:34 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:34.491+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:34.925 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:34 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:34.491+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:04.588320+0000 front 2024-03-24T12:46:04.588417+0000 (oldest deadline 2024-03-24T12:46:28.688254+0000) 2024-03-24T12:46:34.925 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:34 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:34.492+0000 7fb22c30f700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24430.0:2183 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-03-24T12:46:35.512 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:35 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:35.202+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:35.512 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:35 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:35.202+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.652310+0000 front 2024-03-24T12:45:54.652377+0000 (oldest deadline 2024-03-24T12:46:20.552263+0000) 2024-03-24T12:46:35.513 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:35 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:35.202+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:05.853303+0000 front 2024-03-24T12:46:05.853223+0000 (oldest deadline 2024-03-24T12:46:28.653656+0000) 2024-03-24T12:46:35.924 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:35 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:35.516+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:35.924 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:35 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:35.516+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:35.924 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:35 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:35.516+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:04.588320+0000 front 2024-03-24T12:46:04.588417+0000 (oldest deadline 2024-03-24T12:46:28.688254+0000) 2024-03-24T12:46:35.925 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:35 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:35.517+0000 7fb22c30f700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24430.0:2183 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-03-24T12:46:36.560 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:36 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:36.210+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:36.560 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:36 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:36.210+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.652310+0000 front 2024-03-24T12:45:54.652377+0000 (oldest deadline 2024-03-24T12:46:20.552263+0000) 2024-03-24T12:46:36.560 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:36 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:36.210+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:05.853303+0000 front 2024-03-24T12:46:05.853223+0000 (oldest deadline 2024-03-24T12:46:28.653656+0000) 2024-03-24T12:46:36.924 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:36 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:36.564+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:36.924 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:36 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:36.564+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:36.924 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:36 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:36.564+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:04.588320+0000 front 2024-03-24T12:46:04.588417+0000 (oldest deadline 2024-03-24T12:46:28.688254+0000) 2024-03-24T12:46:36.925 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:36 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:36.565+0000 7fb22c30f700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24430.0:2183 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-03-24T12:46:37.424 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:37 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:37.170+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:37.425 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:37 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:37.170+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.652310+0000 front 2024-03-24T12:45:54.652377+0000 (oldest deadline 2024-03-24T12:46:20.552263+0000) 2024-03-24T12:46:37.425 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:37 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:37.170+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:05.853303+0000 front 2024-03-24T12:46:05.853223+0000 (oldest deadline 2024-03-24T12:46:28.653656+0000) 2024-03-24T12:46:37.425 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:37 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:37.170+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6830 osd.3 since back 2024-03-24T12:46:10.354689+0000 front 2024-03-24T12:46:10.354671+0000 (oldest deadline 2024-03-24T12:46:36.254562+0000) 2024-03-24T12:46:37.924 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:37 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:37.519+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:37.924 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:37 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:37.519+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:37.925 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:37 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:37.519+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:04.588320+0000 front 2024-03-24T12:46:04.588417+0000 (oldest deadline 2024-03-24T12:46:28.688254+0000) 2024-03-24T12:46:37.925 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:37 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:37.519+0000 7fb22c30f700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24430.0:2183 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-03-24T12:46:38.424 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:38 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:38.144+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.551723+0000 front 2024-03-24T12:45:50.551937+0000 (oldest deadline 2024-03-24T12:46:14.651736+0000) 2024-03-24T12:46:38.425 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:38 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:38.144+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.652310+0000 front 2024-03-24T12:45:54.652377+0000 (oldest deadline 2024-03-24T12:46:20.552263+0000) 2024-03-24T12:46:38.425 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:38 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:38.144+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:05.853303+0000 front 2024-03-24T12:46:05.853223+0000 (oldest deadline 2024-03-24T12:46:28.653656+0000) 2024-03-24T12:46:38.425 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:38 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6[51014]: 2024-03-24T12:46:38.144+0000 7f1d43d6d700 -1 osd.6 793 heartbeat_check: no reply from 172.21.15.6:6830 osd.3 since back 2024-03-24T12:46:10.354689+0000 front 2024-03-24T12:46:10.354671+0000 (oldest deadline 2024-03-24T12:46:36.254562+0000) 2024-03-24T12:46:38.858 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:38 smithi090 podman[79712]: 2024-03-24 12:46:38.575854807 +0000 UTC m=+5.448088963 container died e03e219bc54e3b8613b70401c4687f7454aa6ca799d6cffaa5334e549207ce4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T12:46:38.858 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:38 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:38.479+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:38.858 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:38 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:38.479+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:38.859 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:38 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:38.479+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:04.588320+0000 front 2024-03-24T12:46:04.588417+0000 (oldest deadline 2024-03-24T12:46:28.688254+0000) 2024-03-24T12:46:38.859 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:38 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:38.479+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6830 osd.3 since back 2024-03-24T12:46:12.789273+0000 front 2024-03-24T12:46:12.789359+0000 (oldest deadline 2024-03-24T12:46:38.089179+0000) 2024-03-24T12:46:38.859 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:38 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:38.479+0000 7fb22c30f700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24430.0:2183 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-03-24T12:46:39.174 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:38 smithi090 podman[79712]: 2024-03-24 12:46:38.969837161 +0000 UTC m=+5.842071309 container cleanup e03e219bc54e3b8613b70401c4687f7454aa6ca799d6cffaa5334e549207ce4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-24T12:46:39.174 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:38 smithi090 bash[79712]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6 2024-03-24T12:46:39.475 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:39 smithi090 podman[79735]: 2024-03-24 12:46:39.196181525 +0000 UTC m=+0.622388639 container remove e03e219bc54e3b8613b70401c4687f7454aa6ca799d6cffaa5334e549207ce4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-24T12:46:39.779 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:39 smithi090 podman[79832]: 2024-03-24 12:46:39.414127168 +0000 UTC m=+0.017850545 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-24T12:46:39.780 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:39 smithi090 podman[79832]: 2024-03-24 12:46:39.564098665 +0000 UTC m=+0.167822073 container create 2667621b786800f996193b64aeb883039d34b786142c23c2b81cbf51133dd6a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T12:46:39.780 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:39 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:39.480+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:39.780 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:39 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:39.480+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:39.780 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:39 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:39.480+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:04.588320+0000 front 2024-03-24T12:46:04.588417+0000 (oldest deadline 2024-03-24T12:46:28.688254+0000) 2024-03-24T12:46:39.780 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:39 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:39.480+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6830 osd.3 since back 2024-03-24T12:46:12.789273+0000 front 2024-03-24T12:46:12.789359+0000 (oldest deadline 2024-03-24T12:46:38.089179+0000) 2024-03-24T12:46:39.780 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:39 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:39.480+0000 7fb22c30f700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24430.0:2183 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-03-24T12:46:40.070 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:39 smithi090 podman[79832]: 2024-03-24 12:46:39.784749669 +0000 UTC m=+0.388473052 container init 2667621b786800f996193b64aeb883039d34b786142c23c2b81cbf51133dd6a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T12:46:40.071 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:39 smithi090 podman[79832]: 2024-03-24 12:46:39.790169472 +0000 UTC m=+0.393892857 container start 2667621b786800f996193b64aeb883039d34b786142c23c2b81cbf51133dd6a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-24T12:46:40.071 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:39 smithi090 podman[79832]: 2024-03-24 12:46:39.859714343 +0000 UTC m=+0.463437740 container attach 2667621b786800f996193b64aeb883039d34b786142c23c2b81cbf51133dd6a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-24T12:46:40.358 INFO:journalctl@ceph.osd.6.smithi090.stdout:Mar 24 12:46:40 smithi090 podman[79832]: 2024-03-24 12:46:40.076018545 +0000 UTC m=+0.679741938 container died 2667621b786800f996193b64aeb883039d34b786142c23c2b81cbf51133dd6a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b) 2024-03-24T12:46:40.608 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:40 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:40.492+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:40.608 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:40 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:40.492+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:40.608 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:40 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:40.492+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:04.588320+0000 front 2024-03-24T12:46:04.588417+0000 (oldest deadline 2024-03-24T12:46:28.688254+0000) 2024-03-24T12:46:40.608 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:40 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:40.492+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6830 osd.3 since back 2024-03-24T12:46:12.789273+0000 front 2024-03-24T12:46:12.789359+0000 (oldest deadline 2024-03-24T12:46:38.089179+0000) 2024-03-24T12:46:40.609 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:40 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:40.492+0000 7fb22c30f700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24430.0:2183 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-03-24T12:46:40.629 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.6.service' 2024-03-24T12:46:40.670 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T12:46:40.670 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-24T12:46:40.670 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-24T12:46:40.670 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.7 2024-03-24T12:46:40.924 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:40 smithi090 systemd[1]: Stopping Ceph osd.7 for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:46:41.424 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:41 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:41.035+0000 7fb22fd1b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T12:46:41.424 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:41 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:41.035+0000 7fb22fd1b700 -1 osd.7 793 *** Got signal Terminated *** 2024-03-24T12:46:41.424 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:41 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:41.035+0000 7fb22fd1b700 -1 osd.7 793 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-24T12:46:41.924 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:41 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:41.450+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:41.925 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:41 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:41.450+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:41.925 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:41 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:41.450+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:04.588320+0000 front 2024-03-24T12:46:04.588417+0000 (oldest deadline 2024-03-24T12:46:28.688254+0000) 2024-03-24T12:46:41.925 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:41 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:41.450+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6830 osd.3 since back 2024-03-24T12:46:12.789273+0000 front 2024-03-24T12:46:12.789359+0000 (oldest deadline 2024-03-24T12:46:38.089179+0000) 2024-03-24T12:46:41.925 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:41 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:41.450+0000 7fb22c30f700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24430.0:2183 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-03-24T12:46:42.674 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:42.425+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:42.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:42.425+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:42.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:42.425+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:04.588320+0000 front 2024-03-24T12:46:04.588417+0000 (oldest deadline 2024-03-24T12:46:28.688254+0000) 2024-03-24T12:46:42.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:42.425+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6830 osd.3 since back 2024-03-24T12:46:12.789273+0000 front 2024-03-24T12:46:12.789359+0000 (oldest deadline 2024-03-24T12:46:38.089179+0000) 2024-03-24T12:46:42.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:42.425+0000 7fb22c30f700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24430.0:2183 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-03-24T12:46:42.675 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:46:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:46:42.474Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.6:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.6:8765: connect: connection refused" 2024-03-24T12:46:42.675 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:46:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:46:42.476Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.6:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.6:8765: connect: connection refused" 2024-03-24T12:46:42.675 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:46:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:46:42.476Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.6:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.6:8765: connect: connection refused" 2024-03-24T12:46:42.675 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:46:42 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:46:42.477Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.6:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.6:8765: connect: connection refused" 2024-03-24T12:46:43.381 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:43 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:43.385+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:43.381 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:43 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:43.385+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:43.381 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:43 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:43.385+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:04.588320+0000 front 2024-03-24T12:46:04.588417+0000 (oldest deadline 2024-03-24T12:46:28.688254+0000) 2024-03-24T12:46:43.381 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:43 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:43.385+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6830 osd.3 since back 2024-03-24T12:46:12.789273+0000 front 2024-03-24T12:46:12.789359+0000 (oldest deadline 2024-03-24T12:46:38.089179+0000) 2024-03-24T12:46:43.381 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:43 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:43.385+0000 7fb22c30f700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24430.0:2183 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-03-24T12:46:44.674 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:44 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:44.370+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:44.674 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:44 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:44.370+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:44.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:44 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:44.370+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:04.588320+0000 front 2024-03-24T12:46:04.588417+0000 (oldest deadline 2024-03-24T12:46:28.688254+0000) 2024-03-24T12:46:44.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:44 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:44.370+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6830 osd.3 since back 2024-03-24T12:46:12.789273+0000 front 2024-03-24T12:46:12.789359+0000 (oldest deadline 2024-03-24T12:46:38.089179+0000) 2024-03-24T12:46:44.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:44 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:44.370+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.90:6804 osd.4 since back 2024-03-24T12:46:18.089788+0000 front 2024-03-24T12:46:18.089660+0000 (oldest deadline 2024-03-24T12:46:43.989635+0000) 2024-03-24T12:46:44.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:44 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:44.370+0000 7fb22c30f700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24430.0:2183 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-03-24T12:46:45.674 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:45 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:45.379+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6806 osd.0 since back 2024-03-24T12:45:50.586171+0000 front 2024-03-24T12:45:50.586263+0000 (oldest deadline 2024-03-24T12:46:13.486370+0000) 2024-03-24T12:46:45.674 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:45 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:45.379+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6814 osd.1 since back 2024-03-24T12:45:54.587461+0000 front 2024-03-24T12:45:54.587324+0000 (oldest deadline 2024-03-24T12:46:20.487321+0000) 2024-03-24T12:46:45.674 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:45 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:45.379+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6822 osd.2 since back 2024-03-24T12:46:04.588320+0000 front 2024-03-24T12:46:04.588417+0000 (oldest deadline 2024-03-24T12:46:28.688254+0000) 2024-03-24T12:46:45.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:45 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:45.379+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.6:6830 osd.3 since back 2024-03-24T12:46:12.789273+0000 front 2024-03-24T12:46:12.789359+0000 (oldest deadline 2024-03-24T12:46:38.089179+0000) 2024-03-24T12:46:45.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:45 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:45.379+0000 7fb22c30f700 -1 osd.7 793 heartbeat_check: no reply from 172.21.15.90:6804 osd.4 since back 2024-03-24T12:46:18.089788+0000 front 2024-03-24T12:46:18.089660+0000 (oldest deadline 2024-03-24T12:46:43.989635+0000) 2024-03-24T12:46:45.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:45 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7[57367]: 2024-03-24T12:46:45.379+0000 7fb22c30f700 -1 osd.7 793 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24430.0:2183 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e793) 2024-03-24T12:46:46.764 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:46 smithi090 podman[79963]: 2024-03-24 12:46:46.482341427 +0000 UTC m=+5.577037365 container died 47091026505ded6d56eed521d3594dd10e392103bf8de787693f5d181f3d4719 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7, RELEASE=squid-db0330b, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-24T12:46:47.097 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:46 smithi090 podman[79963]: 2024-03-24 12:46:46.851754636 +0000 UTC m=+5.946450580 container cleanup 47091026505ded6d56eed521d3594dd10e392103bf8de787693f5d181f3d4719 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-24T12:46:47.098 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:46 smithi090 bash[79963]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7 2024-03-24T12:46:47.415 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:47 smithi090 podman[79986]: 2024-03-24 12:46:47.102894679 +0000 UTC m=+0.619787268 container remove 47091026505ded6d56eed521d3594dd10e392103bf8de787693f5d181f3d4719 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7, ceph=True, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-24T12:46:47.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:47 smithi090 podman[80078]: 2024-03-24 12:46:47.323679774 +0000 UTC m=+0.017973617 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-24T12:46:47.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:47 smithi090 podman[80078]: 2024-03-24 12:46:47.429002263 +0000 UTC m=+0.123296174 container create 59650e637d4a279f3e6270b45f7a8e6e06603349f2242448e5c59d5cce1bfd8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-24T12:46:47.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:47 smithi090 podman[80078]: 2024-03-24 12:46:47.623186702 +0000 UTC m=+0.317480545 container init 59650e637d4a279f3e6270b45f7a8e6e06603349f2242448e5c59d5cce1bfd8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7-deactivate, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-24T12:46:47.675 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:47 smithi090 podman[80078]: 2024-03-24 12:46:47.629020606 +0000 UTC m=+0.323314452 container start 59650e637d4a279f3e6270b45f7a8e6e06603349f2242448e5c59d5cce1bfd8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-24T12:46:48.174 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:47 smithi090 podman[80078]: 2024-03-24 12:46:47.706456783 +0000 UTC m=+0.400750634 container attach 59650e637d4a279f3e6270b45f7a8e6e06603349f2242448e5c59d5cce1bfd8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-24T12:46:48.174 INFO:journalctl@ceph.osd.7.smithi090.stdout:Mar 24 12:46:47 smithi090 podman[80078]: 2024-03-24 12:46:47.914606149 +0000 UTC m=+0.608899992 container died 59650e637d4a279f3e6270b45f7a8e6e06603349f2242448e5c59d5cce1bfd8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-24T12:46:48.476 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@osd.7.service' 2024-03-24T12:46:48.516 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T12:46:48.517 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-24T12:46:48.517 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-03-24T12:46:48.517 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@rgw.foo.a 2024-03-24T12:46:48.875 INFO:journalctl@ceph.rgw.foo.a.smithi006.stdout:Mar 24 12:46:48 smithi006 systemd[1]: Stopping Ceph rgw.foo.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:46:49.207 INFO:journalctl@ceph.rgw.foo.a.smithi006.stdout:Mar 24 12:46:48 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-rgw-foo-a[69936]: 2024-03-24T12:46:48.878+0000 7fac3e140700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-24T12:46:49.207 INFO:journalctl@ceph.rgw.foo.a.smithi006.stdout:Mar 24 12:46:48 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-rgw-foo-a[69936]: 2024-03-24T12:46:48.879+0000 7fac4c9b0ac0 -1 shutting down 2024-03-24T12:46:59.180 INFO:journalctl@ceph.rgw.foo.a.smithi006.stdout:Mar 24 12:46:58 smithi006 bash[161039]: time="2024-03-24T12:46:58Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-03-24T12:46:59.180 INFO:journalctl@ceph.rgw.foo.a.smithi006.stdout:Mar 24 12:46:58 smithi006 podman[161039]: 2024-03-24 12:46:58.902068848 +0000 UTC m=+10.177834532 container died f208de7bada3e5f1802ecd744051240e1b954900b543e6206ec627a45e1740ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-rgw-foo-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b) 2024-03-24T12:46:59.457 INFO:journalctl@ceph.rgw.foo.a.smithi006.stdout:Mar 24 12:46:59 smithi006 podman[161039]: 2024-03-24 12:46:59.275624444 +0000 UTC m=+10.551390130 container cleanup f208de7bada3e5f1802ecd744051240e1b954900b543e6206ec627a45e1740ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-rgw-foo-a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, org.label-schema.build-date=20240307, ceph=True) 2024-03-24T12:46:59.457 INFO:journalctl@ceph.rgw.foo.a.smithi006.stdout:Mar 24 12:46:59 smithi006 bash[161039]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-rgw-foo-a 2024-03-24T12:46:59.862 DEBUG:teuthology.orchestra.run.smithi006:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@rgw.foo.a.service' 2024-03-24T12:46:59.901 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T12:46:59.902 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-03-24T12:46:59.902 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-03-24T12:46:59.902 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@prometheus.a 2024-03-24T12:47:00.195 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:46:59 smithi090 systemd[1]: Stopping Ceph prometheus.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:47:00.461 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:47:00 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:47:00.200Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-03-24T12:47:00.461 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:47:00 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:47:00.201Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-03-24T12:47:00.461 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:47:00 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:47:00.201Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-03-24T12:47:00.461 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:47:00 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:47:00.201Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-03-24T12:47:00.461 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:47:00 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:47:00.201Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-03-24T12:47:00.461 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:47:00 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:47:00.201Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-03-24T12:47:00.462 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:47:00 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:47:00.201Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-03-24T12:47:00.462 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:47:00 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:47:00.201Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-03-24T12:47:00.462 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:47:00 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:47:00.201Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-03-24T12:47:00.462 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:47:00 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:47:00.202Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-03-24T12:47:00.462 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:47:00 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:47:00.202Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-03-24T12:47:00.462 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:47:00 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a[68177]: ts=2024-03-24T12:47:00.202Z caller=main.go:1147 level=info msg="See you next time!" 2024-03-24T12:47:00.462 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:47:00 smithi090 podman[80209]: 2024-03-24 12:47:00.211185913 +0000 UTC m=+0.111390123 container died 72dfed8906bfef48481814a6886e4434cb6cb8c00aaf5d4c6f64d039fdc5302d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T12:47:00.819 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:47:00 smithi090 podman[80209]: 2024-03-24 12:47:00.564621851 +0000 UTC m=+0.464826056 container cleanup 72dfed8906bfef48481814a6886e4434cb6cb8c00aaf5d4c6f64d039fdc5302d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-24T12:47:00.819 INFO:journalctl@ceph.prometheus.a.smithi090.stdout:Mar 24 12:47:00 smithi090 bash[80209]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-prometheus-a 2024-03-24T12:47:00.977 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@prometheus.a.service' 2024-03-24T12:47:01.014 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-24T12:47:01.014 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-03-24T12:47:01.014 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-24T12:47:01.200 INFO:teuthology.orchestra.run.smithi006.stdout:Deleting cluster with fsid: 66d0ad74-e9d6-11ee-95cd-87774f69a715 2024-03-24T12:47:03.018 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:47:02 smithi006 systemd[1]: Stopping Ceph alertmanager.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:47:03.018 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:47:02 smithi006 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a[80935]: ts=2024-03-24T12:47:02.992Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-03-24T12:47:03.427 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:47:03 smithi006 podman[161560]: 2024-03-24 12:47:03.022974729 +0000 UTC m=+0.110425719 container died 52006839997a02f5b023ac7c4a78a7cda3c324576a621d404782cfbcbdc3ec4e (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T12:47:03.707 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:47:03 smithi006 podman[161560]: 2024-03-24 12:47:03.522728189 +0000 UTC m=+0.610179189 container cleanup 52006839997a02f5b023ac7c4a78a7cda3c324576a621d404782cfbcbdc3ec4e (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T12:47:03.707 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:47:03 smithi006 bash[161560]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a 2024-03-24T12:47:04.144 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:47:03 smithi006 podman[161579]: 2024-03-24 12:47:03.732009467 +0000 UTC m=+0.702299284 container remove 52006839997a02f5b023ac7c4a78a7cda3c324576a621d404782cfbcbdc3ec4e (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-24T12:47:04.144 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:47:03 smithi006 podman[161579]: 2024-03-24 12:47:03.807794379 +0000 UTC m=+0.778084216 volume remove 17adfea2088e0620e0aff02152fc0bdd6823cbd15a3533fc30e58c510e31366c 2024-03-24T12:47:04.144 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:47:03 smithi006 systemd[1]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-03-24T12:47:04.144 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:47:03 smithi006 systemd[1]: Stopped Ceph alertmanager.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:47:04.144 INFO:journalctl@ceph.alertmanager.a.smithi006.stdout:Mar 24 12:47:03 smithi006 systemd[1]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@alertmanager.a.service: Consumed 2.468s CPU time. 2024-03-24T12:47:04.402 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:47:04 smithi006 systemd[1]: Stopping Ceph node-exporter.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:47:04.707 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:47:04 smithi006 podman[161698]: 2024-03-24 12:47:04.407343206 +0000 UTC m=+0.120708284 container died 77aa8ce8cb41f28d854c4fae4af2e4aaea01773712e4ad32349a9c2d4e991bad (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-24T12:47:05.207 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:47:04 smithi006 podman[161698]: 2024-03-24 12:47:04.818474291 +0000 UTC m=+0.531839374 container cleanup 77aa8ce8cb41f28d854c4fae4af2e4aaea01773712e4ad32349a9c2d4e991bad (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-24T12:47:05.207 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:47:04 smithi006 bash[161698]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a 2024-03-24T12:47:05.207 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:47:05 smithi006 podman[161717]: 2024-03-24 12:47:05.011053336 +0000 UTC m=+0.604241087 container remove 77aa8ce8cb41f28d854c4fae4af2e4aaea01773712e4ad32349a9c2d4e991bad (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-24T12:47:05.207 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:47:05 smithi006 systemd[1]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-03-24T12:47:05.624 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:47:05 smithi006 systemd[1]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-03-24T12:47:05.624 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:47:05 smithi006 systemd[1]: Stopped Ceph node-exporter.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:47:05.624 INFO:journalctl@ceph.node-exporter.a.smithi006.stdout:Mar 24 12:47:05 smithi006 systemd[1]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@node-exporter.a.service: Consumed 5.315s CPU time. 2024-03-24T12:47:05.853 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-24T12:47:06.046 INFO:teuthology.orchestra.run.smithi090.stdout:Deleting cluster with fsid: 66d0ad74-e9d6-11ee-95cd-87774f69a715 2024-03-24T12:47:07.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:47:07 smithi090 systemd[1]: Stopping Ceph iscsi.iscsi.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:47:07.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:47:07 smithi090 ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a[62904]: debug Shutdown received 2024-03-24T12:47:17.826 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:47:17 smithi090 bash[80643]: time="2024-03-24T12:47:17Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-03-24T12:47:17.826 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:47:17 smithi090 podman[80643]: 2024-03-24 12:47:17.550252287 +0000 UTC m=+10.131054029 container died 4184f9d18fedbd11681648594dc01a6bc78f9f232bd02a2acc1412491c7fea3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-24T12:47:18.159 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:47:17 smithi090 podman[80643]: 2024-03-24 12:47:17.921469176 +0000 UTC m=+10.502270916 container cleanup 4184f9d18fedbd11681648594dc01a6bc78f9f232bd02a2acc1412491c7fea3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-24T12:47:18.159 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:47:17 smithi090 bash[80643]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a 2024-03-24T12:47:18.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:47:18 smithi090 podman[80669]: 2024-03-24 12:47:18.164173973 +0000 UTC m=+0.613066359 container remove 4184f9d18fedbd11681648594dc01a6bc78f9f232bd02a2acc1412491c7fea3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-iscsi-iscsi-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-24T12:47:18.425 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:47:18 smithi090 systemd[1]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-03-24T12:47:18.425 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:47:18 smithi090 systemd[1]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-03-24T12:47:18.425 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:47:18 smithi090 systemd[1]: Stopped Ceph iscsi.iscsi.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:47:18.425 INFO:journalctl@ceph.iscsi.iscsi.a.smithi090.stdout:Mar 24 12:47:18 smithi090 systemd[1]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@iscsi.iscsi.a.service: Consumed 4.347s CPU time. 2024-03-24T12:47:20.175 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:47:19 smithi090 systemd[1]: Stopping Ceph grafana.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:47:20.175 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:47:20 smithi090 podman[80945]: 2024-03-24 12:47:20.011599409 +0000 UTC m=+0.104560942 container died d81656c62fe7fb48f5fdabb78de8d89df9dd4f8660236481bdb5a0ed75795d98 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a, io.k8s.display-name=Red Hat Universal Base Image 8, vcs-type=git, vendor=Red Hat, Inc., name=ubi8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.tags=base rhel8, summary=Grafana Container configured for Ceph mgr/dashboard integration, build-date=2022-04-29T11:48:11.055897, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.buildah.version=1.32.0, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., architecture=x86_64, release=239.1651231664, distribution-scope=public, description=Ceph Grafana Container, io.openshift.expose-services=, maintainer=Paul Cuzner , vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, version=8.5, com.redhat.component=ubi8-container) 2024-03-24T12:47:20.511 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:47:20 smithi090 podman[80945]: 2024-03-24 12:47:20.33960623 +0000 UTC m=+0.432567762 container cleanup d81656c62fe7fb48f5fdabb78de8d89df9dd4f8660236481bdb5a0ed75795d98 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a, com.redhat.component=ubi8-container, io.openshift.tags=base rhel8, maintainer=Paul Cuzner , com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, version=8.5, vcs-type=git, vendor=Red Hat, Inc., description=Ceph Grafana Container, io.buildah.version=1.32.0, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, distribution-scope=public, release=239.1651231664, summary=Grafana Container configured for Ceph mgr/dashboard integration, architecture=x86_64, io.k8s.display-name=Red Hat Universal Base Image 8, build-date=2022-04-29T11:48:11.055897, io.openshift.expose-services=, name=ubi8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664) 2024-03-24T12:47:20.511 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:47:20 smithi090 bash[80945]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a 2024-03-24T12:47:20.924 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:47:20 smithi090 podman[80964]: 2024-03-24 12:47:20.515880225 +0000 UTC m=+0.507132096 container remove d81656c62fe7fb48f5fdabb78de8d89df9dd4f8660236481bdb5a0ed75795d98 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-grafana-a, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., distribution-scope=public, io.openshift.expose-services=, maintainer=Paul Cuzner , name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.k8s.display-name=Red Hat Universal Base Image 8, version=8.5, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.tags=base rhel8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.buildah.version=1.32.0, description=Ceph Grafana Container, release=239.1651231664, architecture=x86_64, vendor=Red Hat, Inc., com.redhat.component=ubi8-container, build-date=2022-04-29T11:48:11.055897) 2024-03-24T12:47:20.925 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:47:20 smithi090 systemd[1]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-03-24T12:47:20.925 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:47:20 smithi090 systemd[1]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@grafana.a.service: Failed with result 'exit-code'. 2024-03-24T12:47:20.925 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:47:20 smithi090 systemd[1]: Stopped Ceph grafana.a for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:47:20.925 INFO:journalctl@ceph.grafana.a.smithi090.stdout:Mar 24 12:47:20 smithi090 systemd[1]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@grafana.a.service: Consumed 17.934s CPU time. 2024-03-24T12:47:21.423 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:47:20 smithi090 systemd[1]: Stopping Ceph node-exporter.b for 66d0ad74-e9d6-11ee-95cd-87774f69a715... 2024-03-24T12:47:21.423 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:47:21 smithi090 podman[81081]: 2024-03-24 12:47:21.190548524 +0000 UTC m=+0.110436243 container died f2c2d5ff5c48d84f23206dd7156f71519e0a07714ec57a30e5a6fa592a68e0df (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-24T12:47:21.674 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:47:21 smithi090 podman[81081]: 2024-03-24 12:47:21.520025849 +0000 UTC m=+0.439913567 container cleanup f2c2d5ff5c48d84f23206dd7156f71519e0a07714ec57a30e5a6fa592a68e0df (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-24T12:47:21.674 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:47:21 smithi090 bash[81081]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b 2024-03-24T12:47:22.070 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:47:21 smithi090 podman[81100]: 2024-03-24 12:47:21.704627643 +0000 UTC m=+0.517739821 container remove f2c2d5ff5c48d84f23206dd7156f71519e0a07714ec57a30e5a6fa592a68e0df (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-24T12:47:22.070 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:47:21 smithi090 systemd[1]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-03-24T12:47:22.070 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:47:21 smithi090 systemd[1]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-03-24T12:47:22.071 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:47:21 smithi090 systemd[1]: Stopped Ceph node-exporter.b for 66d0ad74-e9d6-11ee-95cd-87774f69a715. 2024-03-24T12:47:22.071 INFO:journalctl@ceph.node-exporter.b.smithi090.stdout:Mar 24 12:47:21 smithi090 systemd[1]: ceph-66d0ad74-e9d6-11ee-95cd-87774f69a715@node-exporter.b.service: Consumed 5.406s CPU time. 2024-03-24T12:47:22.542 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-24T12:47:22.577 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-24T12:47:22.606 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-24T12:47:22.609 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617975/remote/smithi006/crash 2024-03-24T12:47:22.609 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/crash -- . 2024-03-24T12:47:22.662 INFO:teuthology.orchestra.run.smithi006.stderr:tar: /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-24T12:47:22.761 INFO:teuthology.orchestra.run.smithi006.stderr:tar: Error is not recoverable: exiting now 2024-03-24T12:47:22.782 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617975/remote/smithi090/crash 2024-03-24T12:47:22.786 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/crash -- . 2024-03-24T12:47:22.823 INFO:teuthology.orchestra.run.smithi090.stderr:tar: /var/lib/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-24T12:47:22.823 INFO:teuthology.orchestra.run.smithi090.stderr:tar: Error is not recoverable: exiting now 2024-03-24T12:47:22.824 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-24T12:47:22.825 DEBUG:teuthology.orchestra.run.smithi006:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-24T12:47:22.860 INFO:teuthology.orchestra.run.smithi006.stdout:2024-03-24T12:20:00.000168+0000 mon.a (mon.0) 2567 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 3 pool(s) do not have an application enabled 2024-03-24T12:47:22.860 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-24T12:47:22.861 DEBUG:teuthology.orchestra.run.smithi006:> sudo egrep '\[SEC\]' /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-24T12:47:22.934 DEBUG:teuthology.orchestra.run.smithi006:> sudo egrep '\[ERR\]' /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-24T12:47:23.006 DEBUG:teuthology.orchestra.run.smithi006:> sudo egrep '\[WRN\]' /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-24T12:47:23.075 INFO:teuthology.orchestra.run.smithi006.stdout:2024-03-24T12:20:00.000168+0000 mon.a (mon.0) 2567 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 3 pool(s) do not have an application enabled 2024-03-24T12:47:23.076 INFO:tasks.cephadm:Compressing logs... 2024-03-24T12:47:23.076 DEBUG:teuthology.orchestra.run.smithi006:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-24T12:47:23.119 DEBUG:teuthology.orchestra.run.smithi090:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-24T12:47:23.143 INFO:teuthology.orchestra.run.smithi006.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose: No such file or directory -- 2024-03-24T12:47:23.143 INFO:teuthology.orchestra.run.smithi006.stderr: /var/log/ceph/cephadm.log 2024-03-24T12:47:23.143 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-mon.a.log 2024-03-24T12:47:23.146 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-24T12:47:23.146 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-client.rgw.foo.a.log 2024-03-24T12:47:23.146 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.1.log 2024-03-24T12:47:23.146 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.3.log 2024-03-24T12:47:23.146 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-mgr.y.log 2024-03-24T12:47:23.146 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.0.log 2024-03-24T12:47:23.152 INFO:teuthology.orchestra.run.smithi090.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-24T12:47:23.152 INFO:teuthology.orchestra.run.smithi090.stderr:: No such file or directory 2024-03-24T12:47:23.153 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.6.log 2024-03-24T12:47:23.153 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-24T12:47:23.153 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.4.log 2024-03-24T12:47:23.153 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.7.log 2024-03-24T12:47:23.153 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.5.log 2024-03-24T12:47:23.153 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.4.log: /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-mgr.x.log 2024-03-24T12:47:23.153 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/tcmu-runner.log 2024-03-24T12:47:23.153 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-24T12:47:23.153 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/tcmu-runner.log: 67.6% -- replaced with /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/tcmu-runner.log.gz 2024-03-24T12:47:23.153 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph.log 2024-03-24T12:47:23.153 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph.cephadm.log: 81.5% -- replaced with /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-24T12:47:23.153 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- 85.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-24T12:47:23.153 INFO:teuthology.orchestra.run.smithi090.stderr: /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-24T12:47:23.153 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5/var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph.log: --verbose -- /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-mon.b.log 2024-03-24T12:47:23.155 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-mgr.y.log: 93.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-24T12:47:23.155 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-24T12:47:23.158 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-volume.log: 87.9% -- replaced with /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-24T12:47:23.158 INFO:teuthology.orchestra.run.smithi090.stderr: 92.2% -- replaced with /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-24T12:47:23.161 INFO:teuthology.orchestra.run.smithi090.stderr: 92.7% -- replaced with /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-mgr.x.log.gz 2024-03-24T12:47:23.163 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph.log 2024-03-24T12:47:23.163 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph.cephadm.log: 89.0% -- replaced with /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-24T12:47:23.164 INFO:teuthology.orchestra.run.smithi006.stderr: 95.2% -- replaced with /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-24T12:47:23.167 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-24T12:47:23.171 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.2.log 2024-03-24T12:47:23.181 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph.log: 93.3%gzip -5 --verbose -- /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-24T12:47:23.191 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-mon.c.log 2024-03-24T12:47:23.191 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-volume.log: -- replaced with /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-24T12:47:23.215 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-mon.c.log: 94.2% -- replaced with /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-03-24T12:47:23.219 INFO:teuthology.orchestra.run.smithi006.stderr: 91.0% -- replaced with /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-24T12:47:23.738 INFO:teuthology.orchestra.run.smithi006.stderr: 91.0% -- replaced with /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-mgr.y.log.gz 2024-03-24T12:47:24.619 INFO:teuthology.orchestra.run.smithi006.stderr: 92.5% -- replaced with /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-mon.c.log.gz 2024-03-24T12:47:24.665 INFO:teuthology.orchestra.run.smithi090.stderr: 92.9% -- replaced with /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-mon.b.log.gz 2024-03-24T12:47:25.920 INFO:teuthology.orchestra.run.smithi006.stderr: 91.9% -- replaced with /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-mon.a.log.gz 2024-03-24T12:47:30.449 INFO:teuthology.orchestra.run.smithi090.stderr: 93.8% -- replaced with /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.6.log.gz 2024-03-24T12:47:30.901 INFO:teuthology.orchestra.run.smithi006.stderr: 93.8% -- replaced with /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.2.log.gz 2024-03-24T12:47:31.342 INFO:teuthology.orchestra.run.smithi090.stderr: 93.9% -- replaced with /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.5.log.gz 2024-03-24T12:47:31.763 INFO:teuthology.orchestra.run.smithi006.stderr: 93.9% -- replaced with /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.1.log.gz 2024-03-24T12:47:31.787 INFO:teuthology.orchestra.run.smithi090.stderr: 94.0% -- replaced with /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.4.log.gz 2024-03-24T12:47:32.150 INFO:teuthology.orchestra.run.smithi090.stderr: 93.9% -- replaced with /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.7.log.gz 2024-03-24T12:47:32.152 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-03-24T12:47:32.152 INFO:teuthology.orchestra.run.smithi090.stderr:real 0m9.018s 2024-03-24T12:47:32.152 INFO:teuthology.orchestra.run.smithi090.stderr:user 0m33.446s 2024-03-24T12:47:32.152 INFO:teuthology.orchestra.run.smithi090.stderr:sys 0m1.215s 2024-03-24T12:47:32.462 INFO:teuthology.orchestra.run.smithi006.stderr: 94.0% -- replaced with /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.0.log.gz 2024-03-24T12:47:32.950 INFO:teuthology.orchestra.run.smithi006.stderr: 94.0% -- replaced with /var/log/ceph/66d0ad74-e9d6-11ee-95cd-87774f69a715/ceph-osd.3.log.gz 2024-03-24T12:47:32.952 INFO:teuthology.orchestra.run.smithi006.stderr: 2024-03-24T12:47:32.952 INFO:teuthology.orchestra.run.smithi006.stderr:real 0m9.822s 2024-03-24T12:47:32.952 INFO:teuthology.orchestra.run.smithi006.stderr:user 0m38.968s 2024-03-24T12:47:32.952 INFO:teuthology.orchestra.run.smithi006.stderr:sys 0m1.291s 2024-03-24T12:47:32.952 INFO:tasks.cephadm:Archiving logs... 2024-03-24T12:47:32.952 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617975/remote/smithi006/log 2024-03-24T12:47:32.954 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-24T12:47:38.141 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617975/remote/smithi090/log 2024-03-24T12:47:38.141 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-24T12:47:42.938 INFO:tasks.cephadm:Removing cluster... 2024-03-24T12:47:42.938 DEBUG:teuthology.orchestra.run.smithi006:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 --force 2024-03-24T12:47:43.136 INFO:teuthology.orchestra.run.smithi006.stdout:Deleting cluster with fsid: 66d0ad74-e9d6-11ee-95cd-87774f69a715 2024-03-24T12:47:43.435 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 66d0ad74-e9d6-11ee-95cd-87774f69a715 --force 2024-03-24T12:47:43.627 INFO:teuthology.orchestra.run.smithi090.stdout:Deleting cluster with fsid: 66d0ad74-e9d6-11ee-95cd-87774f69a715 2024-03-24T12:47:43.920 INFO:tasks.cephadm:Removing cephadm ... 2024-03-24T12:47:43.920 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-24T12:47:43.936 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-24T12:47:43.952 INFO:tasks.cephadm:Teardown complete 2024-03-24T12:47:43.953 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-24T12:47:43.962 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-24T12:47:43.962 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-24T12:47:43.978 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-24T12:47:44.061 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-24T12:47:44.070 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-24T12:47:44.070 DEBUG:teuthology.orchestra.run.smithi006:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-24T12:47:44.073 DEBUG:teuthology.orchestra.run.smithi090:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-24T12:47:44.085 INFO:teuthology.orchestra.run.smithi006.stderr:bash: line 1: ntpq: command not found 2024-03-24T12:47:44.087 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-03-24T12:47:44.089 INFO:teuthology.orchestra.run.smithi006.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-24T12:47:44.089 INFO:teuthology.orchestra.run.smithi006.stdout:=============================================================================== 2024-03-24T12:47:44.089 INFO:teuthology.orchestra.run.smithi006.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 32 +927us[ +921us] +/- 95ms 2024-03-24T12:47:44.089 INFO:teuthology.orchestra.run.smithi006.stdout:^* hv02.front.sepia.ceph.com 3 6 377 27 +40us[ +33us] +/- 75ms 2024-03-24T12:47:44.089 INFO:teuthology.orchestra.run.smithi006.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 -704us[ -704us] +/- 68ms 2024-03-24T12:47:44.089 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T12:47:44.094 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-24T12:47:44.094 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-03-24T12:47:44.095 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 27 +909us[ +889us] +/- 95ms 2024-03-24T12:47:44.095 INFO:teuthology.orchestra.run.smithi090.stdout:^* hv02.front.sepia.ceph.com 3 6 377 24 +33us[ +13us] +/- 75ms 2024-03-24T12:47:44.095 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 -711us[ -731us] +/- 68ms 2024-03-24T12:47:44.095 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-24T12:47:44.095 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-24T12:47:44.111 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-24T12:47:44.112 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-24T12:47:44.181 DEBUG:teuthology.orchestra.run.smithi006:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-24T12:47:44.263 DEBUG:teuthology.orchestra.run.smithi090:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-24T12:47:44.343 DEBUG:teuthology.orchestra.run.smithi006:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-24T12:47:44.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T12:47:44.374 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-24T12:47:44.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T12:47:44.406 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-24T12:47:44.416 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-24T12:47:44.465 INFO:teuthology.task.internal:Duration was 3137.739063 seconds 2024-03-24T12:47:44.465 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-24T12:47:44.510 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-24T12:47:44.511 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-24T12:47:44.513 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-24T12:47:44.554 INFO:teuthology.orchestra.run.smithi006.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-24T12:47:44.557 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-24T12:47:44.959 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-24T12:47:44.959 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi006.front.sepia.ceph.com 2024-03-24T12:47:44.959 DEBUG:teuthology.orchestra.run.smithi006:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-24T12:47:44.979 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi090.front.sepia.ceph.com 2024-03-24T12:47:44.980 DEBUG:teuthology.orchestra.run.smithi090:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-24T12:47:45.007 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-24T12:47:45.007 DEBUG:teuthology.orchestra.run.smithi006:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-24T12:47:45.022 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-24T12:47:45.076 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-24T12:47:45.077 DEBUG:teuthology.orchestra.run.smithi006:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-24T12:47:45.092 DEBUG:teuthology.orchestra.run.smithi090:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-24T12:47:45.537 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-24T12:47:45.547 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-24T12:47:45.547 DEBUG:teuthology.orchestra.run.smithi006:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-24T12:47:45.574 DEBUG:teuthology.orchestra.run.smithi090:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-24T12:47:45.602 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-24T12:47:45.611 DEBUG:teuthology.orchestra.run.smithi006:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-24T12:47:45.617 DEBUG:teuthology.orchestra.run.smithi090:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-24T12:47:45.639 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern = core 2024-03-24T12:47:45.667 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = core 2024-03-24T12:47:45.691 DEBUG:teuthology.orchestra.run.smithi006:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-24T12:47:45.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T12:47:45.709 DEBUG:teuthology.orchestra.run.smithi090:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-24T12:47:45.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-24T12:47:45.745 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-24T12:47:45.756 INFO:teuthology.task.internal:Transferring archived files... 2024-03-24T12:47:45.756 DEBUG:teuthology.misc:Transferring archived files from smithi006:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617975/remote/smithi006 2024-03-24T12:47:45.756 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-24T12:47:45.821 DEBUG:teuthology.misc:Transferring archived files from smithi090:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617975/remote/smithi090 2024-03-24T12:47:45.822 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-24T12:47:45.863 INFO:teuthology.task.internal:Removing archive directory... 2024-03-24T12:47:45.863 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-24T12:47:45.866 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-24T12:47:45.910 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-24T12:47:45.922 INFO:teuthology.task.internal:Not uploading archives. 2024-03-24T12:47:45.923 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-24T12:47:45.998 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-24T12:47:45.998 DEBUG:teuthology.orchestra.run.smithi006:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-24T12:47:46.000 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-24T12:47:46.012 INFO:teuthology.orchestra.run.smithi006.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 24 12:47 /home/ubuntu/cephtest 2024-03-24T12:47:46.015 INFO:teuthology.orchestra.run.smithi090.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 24 12:47 /home/ubuntu/cephtest 2024-03-24T12:47:46.016 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-24T12:47:46.064 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-24T12:47:46.134 INFO:teuthology.nuke:Checking targets against current locks 2024-03-24T12:47:46.157 DEBUG:teuthology.nuke:shortname: smithi006 2024-03-24T12:47:46.157 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-24T12:47:46.176 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi006.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617975', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-24 11:45:08.341573', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD3e/PZKrLah1XZcqTOFzXV/J62SsgJv/6SOlksvbvGVmMmbKA/fKI1isas8RcBuqxEYk12efNzX97pnO5VlY1w='} 2024-03-24T12:47:46.180 DEBUG:teuthology.nuke:shortname: smithi090 2024-03-24T12:47:46.180 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-24T12:47:46.199 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-22_21:08:02-orch-squid-distro-default-smithi/7617975', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-24 11:45:08.340298', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGcb5kFsxPou4rnrqiB0dnqDeVgbhH2uUF4mYd91oTkngMlYK0tYv+Nws3FIshSjS1mFH1Mi+aoqLfB2qx6c67Q='} 2024-03-24T12:47:46.229 INFO:teuthology.orchestra.console.smithi006:Power off 2024-03-24T12:47:46.230 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-24T12:47:46.254 DEBUG:teuthology.orchestra.console.smithi006:power off output: Chassis Power Control: Down/Off 2024-03-24T12:47:46.255 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T12:47:46.270 INFO:teuthology.orchestra.console.smithi090:Power off 2024-03-24T12:47:46.271 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-24T12:47:46.280 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on 2024-03-24T12:47:46.293 DEBUG:teuthology.orchestra.console.smithi090:power off output: Chassis Power Control: Down/Off 2024-03-24T12:47:46.293 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T12:47:46.315 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2024-03-24T12:47:50.280 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T12:47:50.316 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T12:47:50.391 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on 2024-03-24T12:47:50.426 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2024-03-24T12:47:54.396 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T12:47:54.427 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T12:47:54.508 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is off 2024-03-24T12:47:54.537 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2024-03-24T12:47:54.609 INFO:teuthology.orchestra.console.smithi006:Power off completed 2024-03-24T12:47:58.538 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-24T12:47:58.652 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is off 2024-03-24T12:47:58.753 INFO:teuthology.orchestra.console.smithi090:Power off completed 2024-03-24T12:47:58.857 INFO:teuthology.run:Summary data: description: orch/cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} duration: 3137.7390632629395 failure_reason: '"2024-03-24T12:20:00.000168+0000 mon.a (mon.0) 2567 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 3 pool(s) do not have an application enabled" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-03-24T12:47:58.858 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-24T12:47:58.949 INFO:teuthology.run:FAIL